Class determination method of object set and related device

Abstract

The embodiment of the invention discloses a category determination method of an object set and a related device, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. According to the method and the device, identification is not carried out aiming at a single object identifier any more, the multiple object identifiers are divided into the multiple categories, group identification is carried out aiming at the object set comprising the multiple object identifiers, and the identification speed can be improved. Specifically, the method comprises the following steps: determining an object identifier of each category, namely business index values of the same object set at different time, determining the category of the object set according to the difference between sorting results of the business index values, and if the sorting of the business index values of the same object set is greatly changed, determining the category of the object set; if yes, the category of the object set may be a new category, and the object set is focused, so that the object set of the abnormal category is perceived in advance, and the recognition accuracy of the object identifier of the abnormal category is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for determining a category of an object set and a related device. Background Art

[0002] With the development of computer technology, objects can register object identifiers on network platforms and spread multimedia content on network platforms through object identifiers. However, some objects will spread some abnormal multimedia content related to pornography, gambling, drugs, etc. on network platforms through object identifiers, which will cause a relatively bad impact.

[0003] In the related art, considering that the behavior of the object that spreads abnormal multimedia content is coherent and consistent, the category of the object identifier can be identified through machine learning and other models. For example, data related to the object identifier is collected, such as the credit score of the object identifier, the record of the object identifier's historical penalties, etc., and the machine learning and other models are trained to learn the characteristics of various categories of object identifiers, so that in actual applications, based on the data related to the object identifier, it is possible to identify whether the object identifier is abnormal, thereby preventing the abnormal object identifier from continuing to spread abnormal multimedia content.

[0004] However, the above approach can be easily bypassed by objects that spread abnormal multimedia content, resulting in low recognition accuracy for object identification of abnormal categories. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a method for determining the category of an object set and a related device, which are used to improve the recognition accuracy of object identification of abnormal categories.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In one aspect, an embodiment of the present application provides a method for determining a category of an object set, the method comprising:

[0008] Acquire multiple object identifiers, where the object identifiers are carriers of multimedia content transmitted by the objects;

[0009] Clustering the multiple object identifiers to obtain multiple object sets, where different object sets correspond to different categories;

[0010] Determine first business indicator values ​​and second business indicator values ​​respectively corresponding to the multiple object sets, where the first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to represent quantitative data of behaviors generated by the object identifiers at different times;

[0011] Sorting the first business indicator values ​​respectively corresponding to the multiple object sets to obtain a first sorting result, and sorting the second indicator values ​​respectively corresponding to the multiple object sets to obtain a second sorting result;

[0012] According to the difference between the first sorting result and the second sorting result, categories of object sets included in the multiple object sets are determined.

[0013] On the other hand, an embodiment of the present application provides a device for determining a category of a set of objects, the device comprising: an acquisition unit, a clustering unit, a determination unit, and a sorting unit;

[0014] The acquisition unit is used to acquire multiple object identifiers, where the object identifiers are carriers of multimedia content transmitted by the objects;

[0015] The clustering unit is used to cluster the multiple object identifiers to obtain multiple object sets, where different object sets correspond to different categories;

[0016] The determining unit is used to determine first business indicator values ​​and second business indicator values ​​respectively corresponding to the multiple object sets, where the first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to represent quantitative data of behaviors generated by the object identifiers at different times;

[0017] The sorting unit is used to sort the first business indicator values ​​corresponding to the multiple object sets respectively to obtain a first sorting result, and to sort the second indicator values ​​corresponding to the multiple object sets respectively to obtain a second sorting result;

[0018] The determining unit is further configured to determine categories of object sets included in the multiple object sets according to a difference between the first sorting result and the second sorting result.

[0019] On the other hand, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0020] The memory is used to store a computer program and transmit the computer program to the processor;

[0021] The processor is configured to execute the method described in the above aspects according to the instructions in the computer program.

[0022] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the above aspects.

[0023] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method described in the above aspects.

[0024] It can be seen from the above technical solution that multiple object identifiers are obtained, and the object identifiers are carriers of multimedia content transmitted by the object, and the multiple object identifiers are clustered to obtain multiple object sets, so that different object sets correspond to different categories. Although the object identifiers are divided into multiple categories through the object set, it is not known what the specific content of each category is, so as to avoid limiting the recognition of newly emerging categories by classifying based on the characteristics of known categories. In order to mine the newly emerging categories to avoid the low recognition accuracy caused by bypassing and other methods, the first business indicator and the second business indicator corresponding to the multiple object sets are determined, wherein the first business indicator value and the second business indicator value are the values ​​for the same business indicator, and different business indicator values ​​are used to characterize the quantitative data of the behavior generated by the object identifier at different times. The business indicator values ​​from the same time are sorted to obtain the first sorting result and the second sorting result. Since the object identifier of the abnormal type always hopes to get a quick return, the corresponding business indicator value will fluctuate rapidly, so the difference between the first sorting result and the second sorting result can determine the category of the object set included in the multiple object sets.

[0025] Therefore, the present application no longer identifies a single object identifier, but divides multiple object identifiers into multiple categories, and performs group identification on an object set including multiple object identifiers, which can improve the identification speed. Specifically, the object identifier of each category is determined, that is, the business indicator value of the same object set at different times, and the category of the object set is determined based on the difference between the sorting results of the business indicator values. If the sorting of the business indicator values ​​of the same object set changes significantly, the category of the object set may be a newly emerged category, and this type of object set is focused on, so as to perceive the object set of abnormal categories in advance and improve the recognition accuracy of the object identifiers of abnormal categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of an application scenario of the method for determining the category of an object set provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a flow chart of a method for determining a category of an object set provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a new type of mining object set provided by an embodiment of the present application;

[0030] Figure 4 A schematic diagram of dividing multiple object sets provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of determining the category of an object set provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of determining the category of an object set provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of determining the category of an object set provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of a system for determining a category of an object set provided in an embodiment of the present application;

[0035] Fig. 9 A schematic diagram of the structure of a device for determining the category of an object set provided in an embodiment of the present application;

[0036] Fig.10 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0037] Fig.11 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0039] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] In order to identify abnormal multimedia content, in the related technology, considering that the behavior of the object that spreads abnormal multimedia content is coherent and consistent, machine learning and other models will be trained to identify the category of the object identifier, determine whether the object identifier is abnormal, and the possibility that the abnormal object identifier spreads abnormal multimedia content is high. The abnormal object identifier is processed in a timely manner to avoid continuing to spread abnormal multimedia content through the abnormal object identifier. However, this method only identifies a single object identifier, which not only has a slow recognition speed, but also can only identify the abnormal category that has been discovered, and the recognition accuracy is low.

[0041] After research, it was found that abnormal objects usually commit crimes in groups. Abnormal object groups will bypass abnormal detection of machine learning and other models by raising accounts, buying and selling accounts, renting accounts, group control, crowdsourcing, etc. Taking raising accounts as an example, abnormal object groups will establish multiple object identifiers, divide multiple object identifiers into multiple batches, use a batch of object identifiers to spread abnormal multimedia content once, and then use the same batch of object identifiers to spread abnormal multimedia content again after a period of time. In the meantime, other batches of object identifiers are used to spread abnormal multimedia content, so as to destroy the rules that the behavior of the objects is coherent and consistent, thereby bypassing the detection of machine learning and other models, resulting in low recognition accuracy of object identifiers of abnormal categories.

[0042] Based on this, the embodiment of the present application provides a method for determining the category of an object set and a related device. Instead of identifying a single object identifier, multiple object identifiers are divided into multiple categories, and group identification is performed on an object set including multiple object identifiers to improve the identification speed. Specifically, the object identifier of each category is determined, that is, the business indicator value of the same object set at different times, and the category of the object set is determined based on the difference between the sorting results of the business indicator values. If the sorting of the business indicator values ​​of the same object set changes significantly, the category of the object set may be a newly emerged category, and this type of object set is focused on, so as to perceive the object set of abnormal categories in advance and improve the recognition accuracy of the object identifiers of abnormal categories.

[0043] The method for determining the category of an object set provided in the present application can be applied to computer devices with the ability to determine the category of an object set, such as terminal devices and servers. Among them, the terminal device can specifically be a desktop computer, a laptop computer, a smart phone, a tablet computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. Terminal devices and servers can be directly or indirectly connected via wired or wireless communications, and this application does not limit this.

[0044] The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. For example, in an artificial intelligence scenario, based on the new categories of the mined object set, a machine learning model is trained so that the machine learning model can learn more types of object sets, thereby identifying an unknown category of object sets through the trained machine learning model, which can improve the accuracy of recognition.

[0045] To facilitate understanding of the method for determining the category of an object set provided in an embodiment of the present application, the following uses a server as an example in which the execution subject of the method for determining the category of an object set is used to exemplarily introduce the application scenario of the method for determining the category of an object set.

[0046] See also Figure 1 , which is a schematic diagram of an application scenario of the method for determining the category of an object set provided in an embodiment of the present application. Figure 1 As shown, the application scenario includes a server 100, which can be an independent server for determining the category of an object set. After mining the possible categories of the object set, a model that can identify the category of the object set can be trained based on the characteristics of various categories, and the model can be deployed on the server or terminal device corresponding to the product to provide a service for determining the category of the object set; the server 100 can also be a server that provides corresponding services for various products, and the services provided can include, for example, determining the category of the object set. The following is an example of the service provided by the server 100 to determine the category of the object set.

[0047] The server 100 obtains multiple object identifiers, which are carriers of multimedia content disseminated by the object. Taking the live broadcast room account as an example of the object identifier, the anchor object can broadcast through the live broadcast room account and disseminate multimedia content to the audience object in the live broadcast room. Figure 1 As shown, n object identifiers are obtained, where n is an integer greater than 3.

[0048] The server 100 clusters multiple object identifiers to obtain multiple object sets, such as Figure 1 As shown, n object identifiers are clustered to obtain m object sets, where m is a positive integer less than n. Taking object set 1 as an example, object identifier 1 and object identifier n are clustered into one object set. Each object set includes at least one object identifier, so that different object sets correspond to different categories. Although the object identifiers are divided into multiple categories through object sets, the specific content of each category is not known. For example, after 10 object sets are obtained through clustering, it is clear that the object identifiers are currently divided into 10 categories, but it is not known whether each category is a normal category or an abnormal category, or a sub-category such as a drainage category or a pornographic category, thereby avoiding the method of classifying based on the characteristics of known categories to limit the recognition of newly emerging categories.

[0049] In order to mine newly emerged categories to avoid low recognition accuracy caused by bypassing and other means, the server 100 determines the first business indicator and the second business indicator corresponding to the multiple object sets, wherein the first business indicator value and the second business indicator value are the values ​​for the same business indicator, and different business indicator values ​​are used to characterize the quantitative data of the behavior generated by the object identifier at different times. For example, the first business indicator value and the second business indicator value are both for the number of unique visitors (Unique Visitor, UV) entering the live broadcast room, except that the first business indicator value is collected at 8 am and the second business indicator value is collected at 8 pm.

[0050] The server 100 sorts the business indicator values ​​from the same time, that is, sorts the multiple first business indicator values ​​corresponding to the multiple object sets to obtain a first sorting result, and sorts the multiple second business indicator values ​​corresponding to the multiple object sets to obtain a second sorting result. Since abnormal object identifiers always want to be rewarded quickly, the corresponding business indicator value will fluctuate rapidly. Therefore, the server 100 can determine the category of the object set included in the multiple object sets through the difference between the first sorting result and the second sorting result. Figure 1 As shown, the object set m changes from the last to the second from the top, and its fluctuation is large. It may be an abnormal object set. After specific judgment, the object set m is an explosive abnormal account.

[0051] Therefore, the present application no longer identifies a single object identifier, but divides multiple object identifiers into multiple categories, and performs group identification on an object set including multiple object identifiers, which can improve the identification speed. Specifically, the object identifier of each category is determined, that is, the business indicator value of the same object set at different times, and the category of the object set is determined based on the difference between the sorting results of the business indicator values. If the sorting of the business indicator values ​​of the same object set changes significantly, the category of the object set may be a newly emerged category, and this type of object set is focused on, so as to perceive the object set of abnormal categories in advance and improve the recognition accuracy of the object identifiers of abnormal categories.

[0052] The method for determining the category of an object set provided in the embodiment of the present application can be executed by a server. However, in other embodiments of the present application, the terminal device can also have similar functions as the server, thereby executing the method for determining the category of an object set provided in the embodiment of the present application, or the terminal device and the server can jointly execute the method for determining the category of an object set provided in the embodiment of the present application, and this embodiment does not limit this.

[0053] The method for determining the category of an object set provided by the present application is described in detail below through a method embodiment.

[0054] See also Figure 2 , which is a flow chart of the method for determining the category of an object set provided in an embodiment of the present application. For ease of description, the following embodiment is still introduced by taking the execution subject of the method for determining the category of an object set as a server as an example. Figure 2 As shown, the method for determining the category of the object set includes the following steps:

[0055] S201: Acquire multiple object identifiers.

[0056] The object identifier is the carrier of multimedia content disseminated by the object. For example, the object identifier can be a video account that starts live broadcast, a short video account, an e-commerce account, etc., or an account that publishes pictures and texts, an account used for social networking, etc.

[0057] As a possible implementation method, the object identifier can be a live broadcast room identifier, so that subsequent analysis is only performed on the live broadcast room, rather than clustering all identifiers in the platform, such as the identifiers of audience objects. The identification propagation ability of audience objects is low, and their influence on posting abnormal multimedia content in the live broadcast room is small, so they do not need to be clustered, reducing resource usage, thereby reducing the number of clustered object identifiers and improving the recognition speed of object identifier categories.

[0058] It is understandable that in the specific implementation of this application, it involves user-related data such as object identification and object characteristics. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the informed consent or separate consent of the subject of personal information, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0059] S202: Clustering multiple object identifiers to obtain multiple object sets.

[0060] Objects spread multimedia content through object identifiers, such as posting graphic content on social platforms, broadcasting live on live broadcast platforms, etc. By clustering multiple object identifiers, object identifiers with the same characteristics can be classified into one category, that is, object identifiers with high similarity can be classified into the same object set, so that the category of the object set can be identified later, rather than identifying a single object identifier, which improves the recognition speed. Among them, the object set includes at least one object identifier, and different object sets correspond to different categories, that is, object identifiers belonging to different object sets correspond to different categories.

[0061] The embodiment of the present application does not specifically limit the clustering method, and will be explained later using S2021-S2023 as an example, which will not be repeated here.

[0062] S203: Determine first business indicator values ​​and second business indicator values ​​corresponding to a plurality of object sets respectively.

[0063] The first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to characterize the quantitative data of the behavior generated by the object identifier at different times. For example, the first business indicator value is the quantitative data of the behavior generated by each object identifier in each object set included in the multiple object sets for the target business indicator at the first time, and the second business indicator value is the quantitative data of the behavior generated by each object identifier in each object set included in the multiple object sets for the target business indicator at the second time, and the second time is later than the first time. In other words, the first business indicator value and the second business indicator belong to the same type of business indicator value, but the collection time or generation time is different.

[0064] As a possible implementation, the first time corresponding to the first business indicator value may be a time node when the multimedia content is released through the object identifier, and the second time corresponding to the second business indicator value may be a time node at a shorter time after the multimedia content is released through the object identifier. For example, the first business indicator is obtained when the anchor object starts broadcasting, and the second business indicator is obtained five minutes after the anchor object starts broadcasting. Thus, by determining the first time and the second time based on the time node when the multimedia content is released, it can be determined in a targeted manner that the change is caused by the released multimedia content, and the difference between the second time and the first time is small, so it can be discovered in time when the multimedia content does not have a significant impact to avoid its expansion and dissemination.

[0065] Among them, the target business indicator is one of multiple business indicators, such as the website transaction amount (Gross Merchandise Volume, GMV) of the platform on which the object identifier included in the object set publishes multimedia content, the number of visits (Page View, PV) of multimedia content published by the object identifier included in the viewing object set, the number of times the object identifier included in the object set violates the platform regulations, the level of the object identifier included in the object set on the platform, etc.

[0066] S204: sorting the first business indicator values ​​corresponding to the plurality of object sets respectively to obtain a first sorting result, and sorting the second indicator values ​​corresponding to the plurality of object sets respectively to obtain a second sorting result.

[0067] The present application embodiment does not specifically limit the sorting method, and those skilled in the art can set it according to actual needs. Taking reverse sorting as an example, multiple object sets correspond to multiple first business indicator values ​​respectively, and the multiple first business indicator values ​​are reversed to obtain a first sorting result. Similarly, multiple object sets correspond to multiple second business indicator values ​​respectively, and the multiple second indicator values ​​are reversed to obtain a second sorting result.

[0068] The embodiment of the present application does not specifically limit the method for determining the business indicator value, and the following is explained by taking obtaining a representative object identifier from an object set as an example.

[0069] Based on the business indicator, the first representative object identifier corresponding to the first time is determined from each object set included in the multiple object sets, and the business indicator value of the first representative object identifier under the business indicator is used as the first business indicator value. Under different business indicators, the representative object identifiers are different, and the business indicator values ​​are different at different times. Therefore, it is necessary to determine the first representative object identifier from the multiple object identifiers included in each object set based on the business indicator and the first time, that is, based on the business indicator value obtained at the first time, determine the representative first representative object identifier, and use the business indicator value of the first representative object identifier at the first time for the aforementioned business indicator as the first business indicator value. Similarly, based on the business indicator and the second time, the second representative object identifier corresponding to the second time is determined from each object set included in the multiple object sets, and the business indicator value of the second representative object identifier under the business indicator is used as the second business indicator value.

[0070] Therefore, based on the business indicator and time, an object identifier that is representative of the business indicator is selected from the multiple object identifiers included in each object set, and its business indicator value at that time is used as the business indicator value corresponding to the object set. Compared with calculating the average value of the business indicator values ​​of all object identifiers in the object set, the amount of calculation can be reduced and the calculation speed can be improved.

[0071] S205: Determine categories of object sets included in the multiple object sets according to the difference between the first sorting result and the second sorting result.

[0072] Since abnormal type object sets generally hope to get rewards quickly, such as the drainage type object set hopes to achieve a quick drainage effect, the pornographic type object set hopes to attract more viewers to watch the multimedia content it disseminates, etc., their corresponding business indicator values ​​will fluctuate. For example, the GMV of the platform on which the object identifier publishes the multimedia content and the UV of the multimedia content published by the object identifiers included in the viewing object set may grow rapidly, that is, the business indicator value corresponding to the abnormal type object set will fluctuate rapidly. Therefore, the difference between the first sorting result and the second sorting result can be used to determine the category of the object set included in the multiple object sets.

[0073] It can be seen from the above technical solution that multiple object identifiers are obtained, and the object identifiers are carriers of multimedia content transmitted by the object, and the multiple object identifiers are clustered to obtain multiple object sets, so that different object sets correspond to different categories. Although the object identifiers are divided into multiple categories through the object set, it is not known what the specific content of each category is, so as to avoid limiting the recognition of newly emerging categories by classifying based on the characteristics of known categories. In order to mine the newly emerging categories to avoid the low recognition accuracy caused by bypassing and other methods, the first business indicator and the second business indicator corresponding to the multiple object sets are determined, wherein the first business indicator value and the second business indicator value are the values ​​for the same business indicator, and different business indicator values ​​are used to characterize the quantitative data of the behavior generated by the object identifier at different times. The business indicator values ​​from the same time are sorted to obtain the first sorting result and the second sorting result. Since the abnormal object identifier always hopes to get a quick return, the corresponding business indicator value will fluctuate rapidly, so the difference between the first sorting result and the second sorting result can determine the category of the object set included in the multiple object sets.

[0074] Therefore, the present application no longer identifies a single object identifier, but divides multiple object identifiers into multiple categories, and performs group identification on an object set including multiple object identifiers, which can improve the identification speed. Specifically, the object identifier of each category is determined, that is, the business indicator value of the same object set at different times, and the category of the object set is determined based on the difference between the sorting results of the business indicator values. If the sorting of the business indicator values ​​of the same object set changes significantly, the category of the object set may be a newly emerged category, and this type of object set is focused on, so as to perceive the object set of abnormal categories in advance and improve the recognition accuracy of the object identifiers of abnormal categories.

[0075] It should be noted that the method of S201-S205 may not determine the category of each object set included in the multiple object sets, but it will determine the type of object sets that have changed in the multiple object sets, that is, the focus is on digging out types that have not yet been discovered in the object sets. For example, if abnormal objects want to bypass the detection of machine learning models in related technologies, targeted improvements can be made to the way of publishing multimedia content.

[0076] After research, it is found that there are generally two types of object sets that change in multiple object sets. One is the way of publishing multimedia content that is favored by emerging audience objects. This method is generally a normal type. The other is the way of publishing abnormal multimedia content that appears in order to bypass detection by abnormal objects. This method is generally an abnormal type. Figure 3 The two types of recognition are described separately.

[0077] See also Figure 3 , which is a schematic diagram of a new type of mining object set provided in an embodiment of the present application.

[0078] S301: Clustering multiple object identifiers to obtain multiple first to-be-determined object sets.

[0079] The object identifiers belonging to the same first set of objects to be determined correspond to the same category, and the object identifiers belonging to different first sets of objects to be determined correspond to different categories.

[0080] S302: Acquire business data of object identifiers included in multiple first pending object sets under a target dimension.

[0081] The target dimension is a normal dimension or an abnormal dimension, wherein the normal dimension is a data dimension used to mine new types belonging to the normal type, and the abnormal dimension is a data dimension used to mine new types belonging to the abnormal type. Taking the object identifier as an identifier for the live broadcast room as an example, the business data under the normal dimension includes the traffic size, the size of the feature value obtained by the live broadcast room (such as the amount of reward), the number of object identifiers included in the object set, the website transaction amount of the network platform where the live broadcast room is located, the anchor object characteristics and the audience object characteristics. One or more of the business data under the abnormal dimension include the historical violation data of the live broadcast room, historical punishment data, complaint and reporting data, environment blackening (such as the proportion of object identifiers belonging to the abnormal category in the object identifiers in the same environment in the object set) and strategy recall (such as determining the category of the object identifier by setting preset strategies for categories such as pornography and diversion).

[0082] S303: Determine multiple object sets from multiple first to-be-determined object sets according to the business data.

[0083] According to the business data, multiple object sets are determined from the multiple first to-be-determined object sets, so as to determine a representative object set from the multiple first to-be-determined object sets.

[0084] As a possible implementation method, the feature vectors of each first set of objects to be determined can be determined based on the business data, and the comprehensive value of each first set of objects to be determined can be obtained based on the feature vectors, and the comprehensive value can characterize the representativeness of the first set of objects to be determined under the business data. For example, if the business data is the flow rate, the comprehensive value can be the average flow rate of each first set of objects to be determined, so as to select multiple sets of objects with larger flow rates from multiple first sets of objects to be determined.

[0085] If the business data includes multiple dimensions, the feature vectors from the multiple dimensions can be calculated according to a preset weighted ratio to obtain a comprehensive value of the first set of pending objects. For example, if the business data is the flow size and the number of object identifiers included in the object set, each dimension can be assigned 50% importance to obtain a comprehensive value.

[0086] S304: Determine first business indicator values ​​and second business indicator values ​​corresponding to a plurality of object sets respectively.

[0087] S305: sorting the first business indicator values ​​corresponding to the multiple object sets respectively to obtain a first sorting result, and sorting the second indicator values ​​corresponding to the multiple object sets respectively to obtain a second sorting result.

[0088] As a possible implementation method, the sorting results can be multiple. For example, the sorting results can be for different business indicators, such as traffic rankings, audience UV rankings, etc. For another example, the sorting results can also be for different subjects, such as anchor rankings (such as whether the anchor is certified or not, whether it is a top anchor, etc.), live broadcast room rankings (such as live broadcast content, etc.), audience rankings (audience comments, etc.), etc.

[0089] S306: If the difference between the ranking of the first target object set in the first sorting result and the ranking of the first target object set in the second sorting result is greater than the first ranking threshold, and the multiple object sets are determined based on business data under normal dimensions, then the category of the first target object set is a normal category.

[0090] Take one object set among multiple object sets, i.e., the first target object set, as an example. If the first target object set is determined from the object set determined based on the business data under the normal dimension, it means that the first target object set is a highly representative object set under the normal dimension, and it is highly likely to belong to the normal category. If the ranking difference between the first target object set in the two sorting results is greater than the first ranking threshold, it means that the first target object set is a newly appeared category under the normal category.

[0091] As a possible implementation method, after discovering a new category under a normal category, it means that this type can obtain positive benefits, and it can be shared with the target object identifier (such as top anchors, etc.) so that they can quickly master the new method and provide positive output of the business.

[0092] S307: If the difference between the ranking of the first target object set in the first sorting result and the ranking of the first target object set in the second sorting result is greater than a second ranking threshold, and the multiple object sets are determined based on the business data under the abnormal dimension, then the category of the first target object set is an abnormal category.

[0093] If the first target object set is determined from the object set determined based on the business data under the abnormal dimension, it means that the first target object set is a representative object set under the abnormal dimension, and it is likely to belong to the abnormal category. If the ranking difference between the first target object set in the two sorting results is greater than the second ranking threshold, it means that the first target object set is a newly appeared category under the abnormal category. As a possible implementation method, a real-time alarm can be issued for the first target object set belonging to the abnormal category.

[0094] The embodiment of the present application does not specifically limit the size of the first ranking threshold and the second ranking threshold, and those skilled in the art may set them according to actual needs. For example, the first ranking threshold and the second ranking threshold are consistent in size.

[0095] As a possible implementation method, not only can the object identifiers included in the first target object set be processed quickly to prevent them from expanding the distribution range of abnormal multimedia content, but also the new category newly mined under the abnormal category can be analyzed for its characteristics, so that the object set of this category can be quickly identified based on the characteristics in the future, thereby improving the accuracy of identification.

[0096] Therefore, by obtaining business data under the normal dimension or the abnormal dimension, a set of objects is determined from the multiple first sets of pending objects obtained by clustering, which is more likely to belong to the normal category or the abnormal category, and then the set of objects that shows abnormal fluctuations in the sorting results belongs to a new category under the normal category or a new category under the abnormal category.

[0097] As a possible implementation method, the embodiment of the present application provides a specific implementation method of S202, that is, clustering the multiple object identifiers to obtain a specific implementation method of multiple object sets, see S2021-S2023 for details.

[0098] S2021: Obtain relevant data of each object identifier included in the multiple object identifiers.

[0099] The relevant data includes one or more combinations of the behavior data of each object identifier and the multimedia content transmitted through the object identifier. Taking the object identifier as the identifier of the live broadcast room as an example, the behavior data of the object identifier can be the object identifier shared by the anchor object at the start of the broadcast, etc. The multimedia content transmitted through the object identifier can be the live broadcast screen, automatic speech recognition (ASR) for live broadcast, optical character recognition (OCR) for live broadcast, audio of the live broadcast room, body movements of the anchor, etc.

[0100] As a possible implementation method, multimedia content transmitted through object identifiers within a preset time period can be obtained. For example, multimedia content related to the last three live broadcasts. Thus, the category of the object identifier can be identified in combination with historical multimedia content to improve the accuracy of identification.

[0101] As a possible implementation method, the relevant data may also include basic data of each object identifier, such as the device identifier of the device where the object identifier is located, the location of the device where the object identifier is located, the registration time of the object identifier, whether the object identifier is authenticated, etc. Thus, by enriching the relevant data, the accuracy of the subsequent feature vector description of the object identifier is improved.

[0102] As a possible implementation method, the relevant data is obtained based on multiple dimensional sub-data, each dimensional sub-data corresponds to a data category, for example, live broadcast screen, live broadcast ASR, live broadcast OCR, live broadcast room audio, anchor's body movements, etc. are all different dimensional sub-data. In order to avoid the problem of large amount of calculation caused by too much relevant data, the credibility corresponding to the multiple dimensional sub-data can be obtained, and then the relevant data can be obtained based on the credibility corresponding to the multiple dimensional sub-data.

[0103] The higher the credibility of the dimensional sub-data included in the relevant data, the fewer the dimensions of the dimensional sub-data obtained from the multiple dimensional sub-data. Taking the device identifier of the device where the object identifier is located as an example, the credibility of this dimensional sub-data is relatively high. For example, if multiple object identifiers are used to log in through the same device, the probability that the multiple object identifiers belong to an object set is very high. In this case, only this dimensional sub-data can be used as relevant data to obtain the object set through clustering.

[0104] Therefore, the higher the credibility of the dimensional sub-data, the smaller the amount of related data can be. The lower the credibility of the dimensional sub-data, the more the amount of related data can be increased, that is, the related data is composed of multiple dimensional sub-data, so that the subsequent identification feature vector describes the object identification with higher accuracy, thereby improving the accuracy of the object set obtained by subsequent clustering, and further improving the recognition accuracy of the object identification category.

[0105] S2022: Determine the identification feature vector of each object identification based on the relevant data.

[0106] Taking the example that the relevant data includes the behavior data of each object identification and the multimedia content transmitted through the object identification, the identification feature subvectors corresponding to the two types of data can be concatenated into a multi-dimensional identification feature vector. The identification feature vector is used to describe the characteristics of the object identification.

[0107] The embodiment of the present application does not specifically limit the method of extracting the identification feature vector, and the extraction can be performed through a pre-trained machine learning model.

[0108] S2023: Clustering multiple object identifiers according to the identifier feature vectors to obtain multiple object sets.

[0109] The embodiments of the present application do not specifically limit the clustering method, such as K-means clustering method, mean shift clustering method, density-based spatial clustering of applications with noise (DBSCAN), etc.

[0110] Thus, the behavior data of each object identifier and the multimedia content transmitted through the object identifier are clustered to obtain an object set. Compared with only determining object identifiers belonging to the same category based on the same identity, such as abnormal objects using the same device to log in to multiple object identifiers, the embodiment of the present application adds an association method based on content similarity retrieval, which improves the accuracy and recall rate of recognition.

[0111] As a possible implementation, the embodiment of the present application provides a specific implementation of S2023, that is, clustering the multiple object identifiers according to the identification feature vector to obtain the multiple object sets. Specifically, clustering the multiple object identifiers according to the identification feature vector to obtain multiple second pending object sets; determining the overlap between the multiple second specific object sets; merging two second pending object sets whose overlap is greater than the overlap threshold to obtain an object set.

[0112] The object identifiers belonging to the same second undetermined object set correspond to the same category, and the object identifiers belonging to different second undetermined object sets correspond to different categories. The overlap degree is used to identify the overlap degree of the object identifiers included in different second specific object sets. For example, the second specific object set A includes object identifier 1, object identifier 2, object identifier 3 and object identifier 4, and the second specific object set B includes object identifier 1, object identifier 4, object identifier 5 and object identifier 6. Therefore, the overlap degree of the second specific object set A and the second specific object set B is 50%.

[0113] Thus, the overlap of the object identifiers included in the second pending object sets is calculated. If the overlap between the two second pending object sets is high, it means that the number of object identifiers included in the two second pending object sets that overlap is large, and the probability that the two second pending object sets are controlled by the same object group is high. For example, the same object group controls the second pending object set A to publish multimedia content and keeps the second pending object set B silent so that after the second pending object set A is punished, the second pending object set B can be used to publish multimedia content. Therefore, the two second pending object sets with an overlap greater than the overlap threshold can be merged to obtain an object set. This can avoid the object set from being too dispersed, reduce the audit pressure, and improve the accuracy of identification.

[0114] As a possible implementation method, as the number of merged second pending object sets increases, the object set includes a larger number of object identifiers (for the sake of convenience, it is subsequently called an ultra-large object set), which results in the inability to subdivide the category of the object set subsequently determined. For example, it can only be determined that the ultra-large object set is a normal category or an abnormal category, and it cannot be determined whether it belongs to a secondary category such as the pornographic category or the diversion category.

[0115] Based on this, in order to avoid the formation of an extremely large object set, the number of object identifiers included in the object set obtained after the merger can be controlled during the process of merging the two second pending object sets, that is, the number of object identifiers included in the object set is controlled to be less than the upper limit of the number. Thus, the overlap between the object sets obtained based on the second pending object set is lower than the overlap threshold, or the number of object identifiers included in each object set is less than the upper limit of the number.

[0116] Therefore, the object set obtained from the second pending object can not only avoid the formation of an extremely large object set that leads to the inability to accurately determine the category of each object set in the future, but also avoid the object set being too dispersed, reduce the audit pressure, and improve the accuracy of recognition.

[0117] In order to further understand the technical solution provided by the embodiments of the present application, Figure 4 The object identifier is taken as a live broadcast room identifier as an example for explanation.

[0118] See also Figure 4 , which is a schematic diagram of dividing multiple object sets provided in an embodiment of the present application.

[0119] S401: Obtain a live broadcast room identifier from an identifier pool.

[0120] Among them, the platform establishes an identification pool, which includes all the live broadcast room identifications registered on the platform, and each live broadcast room identification, such as the live broadcast room number, can be obtained from the identification pool in sequence.

[0121] S402: Determine whether the corresponding live broadcast room has started live broadcasting according to the live broadcast room identifier. If so, execute S403; if not, end the process.

[0122] S403: Clustering multiple live broadcast room identifiers to obtain multiple second to-be-determined object sets.

[0123] When the live broadcast room starts broadcasting, the live broadcast room logo is used as the main feature, and combined with the multimedia content of the last three live broadcasts, the two types of features are spliced ​​into a multi-dimensional logo feature vector. According to the logo feature vector, multiple live broadcast room logos are clustered to obtain updated multiple second pending object sets.

[0124] As a possible implementation method, the relevant data may include the live broadcast room, anchor content, broadcast equipment, certificates, network, avatar, anchor face and other data.

[0125] S404: Determine whether the number of object identifiers included in the second pending object set is greater than 3. If so, execute S405; if not, end the process.

[0126] S405: Determine the degree of overlap between multiple second specific object sets.

[0127] S406: Merge two second pending object sets whose overlap is greater than the overlap threshold to obtain an object set.

[0128] It should be noted that object sets with object identifiers less than 3 are filtered out because such object sets have no operational significance, and thus there is no need to occupy resources to analyze the categories of such object sets.

[0129] As can be seen from the foregoing, through S201-S205, the categories of various object sets are mined, so before mining new categories, the object sets of abnormal categories can be quickly identified based on the existing categories, so as to perform targeted processing on the object sets of this category.

[0130] Specifically, the credit values ​​of the object identifiers included in the second target object set are obtained, and the average credit value corresponding to the second target object set is determined according to the credit value. If the average credit value is greater than the credit value threshold, the first-level category of the second target object set is determined to be a normal category.

[0131] The second target object set is an object set among multiple object sets. The higher the credit value of an object identifier, the more it complies with the rules of the platform, and the greater the possibility that the multimedia content it publishes is not abnormal multimedia content. Therefore, based on the credit values ​​of each object identifier included in the object set, the average credit value corresponding to the second target object set can be calculated. If the average credit value is greater than the credit value threshold, it means that the second target object set belongs to a high-quality object set and can be added to the whitelist. The corresponding category is the normal category.

[0132] It should be noted that in the embodiment of the present application, the category of the object set can be divided into a primary category and a secondary category. The primary category is used to identify the major category of the object set, such as the normal category and the abnormal category. The secondary category is a subcategory under the primary category, which is used to identify the reason why the object set belongs to the abnormal category. It can be understood that there is no secondary category under the normal category, because generally, it is not concerned with why the object set is normal, but more concerned with why the object set is abnormal, such as whether it belongs to the abnormal category due to drainage reasons, or due to pornographic reasons.

[0133] Obtain abnormal labels of the object identifiers included in the third target object set, determine the abnormal concentration of the third target object set based on the abnormal labels, if the abnormal concentration is greater than the concentration threshold, determine that the first-level category of the third target object set is the abnormal category, obtain relevant data of the object identifiers included in the third target object set, and determine the second-level category of the third target object set based on the relevant data.

[0134] The third target object set is an object set among multiple object sets.

[0135] The higher the abnormal concentration of an object set, the less the object identifiers included in it comply with the rules of the platform, and the greater the possibility that the multimedia content it publishes is abnormal multimedia content. Therefore, the concentration threshold corresponding to the third target object set can be calculated based on the abnormal label of the object identifier. The abnormal label is used to identify whether the object identifier has been abnormal, such as whether it has been punished by the platform, etc. The abnormal label can be determined based on the category of the object set obtained by the aforementioned mining. If the abnormal concentration is greater than the concentration threshold, it means that the third object set needs to be focused on, and its first-level category is the abnormal category. In order to mine the causes of the abnormality, the relevant data of the object identifiers included in the third target object set can be obtained, and the secondary category of the third target object set can be determined based on the relevant data, such as pornographic category, recording and broadcasting category, drainage category, etc.

[0136] Therefore, before mining new categories, the object set of abnormal categories can be quickly identified based on existing categories, so as to carry out targeted processing on the object set of this category.

[0137] In order to further understand the technical solution provided by the embodiments of the present application, Figure 5 The object identifier is taken as a live broadcast room identifier as an example for explanation.

[0138] See also Figure 5 , which is a schematic diagram of determining the category of an object set provided in an embodiment of the present application.

[0139] S501: Calculate the average credit value of each object set.

[0140] S502: Determine whether the average credit value of each object set is greater than 95 points. If so, the first-level category of the object set is a normal category. If not, execute S503.

[0141] S503: Calculate the abnormal concentration of each object set.

[0142] It can be understood that the object sets in S503 are all object sets with an average credit value less than 95 points.

[0143] S504: Determine whether the abnormal concentration of each object set is greater than 50%. If not, the primary category of the object set is a normal category. If so, execute S505.

[0144] S505: Acquire relevant data of the object identification included in each object set.

[0145] It can be understood that the object set in S505 not only has an average credit value less than 95 points, but also has an abnormal concentration greater than 50%.

[0146] S506: Determine the secondary category of each object set according to the relevant data.

[0147] As a possible implementation method, the object set with an abnormal concentration greater than 50% can be manually reviewed. The reviewer determines the secondary category of each object set based on the live broadcast room identification, object set characteristics, live broadcast content, etc. included in the object set.

[0148] As a possible implementation method, in the related art, generally only the category of active object identifiers is identified, such as after multimedia content is disseminated through object identifiers, the category of object identifiers is determined based on the multimedia content. However, since abnormal objects generally commit crimes in groups, they may raise or buy accounts to make the credibility of object identifiers higher, thereby bypassing detection. Based on this, the embodiments of the present application provide two methods to improve the accuracy of identification. They are explained below.

[0149] Method 1: Real-time identification.

[0150] If multimedia content is transmitted through a first target object identifier, at least one abnormal object set in which the first target object identifier is located is obtained; if a second target object identifier included in the abnormal object set is transmitting multimedia content, target multimedia content transmitted by the second target object identifier within a preset time period is collected, so as to determine the category of the second target object identifier according to the target multimedia content.

[0151] Among them, the category of the first target object identifier is an abnormal category, and the abnormal object set is an object set of the abnormal category. It can be understood that, since each clustering will obtain an object set where the first target object identifier is located, the abnormal object set is obtained from the object set where the first target object identifier is located, that is, the object set of the category is the abnormal category. If the second target object identifier included in the abnormal object set is disseminating multimedia content, such as the live broadcast room corresponding to other object identifiers included in the abnormal object set is broadcasting live, then the target multimedia content disseminated by the second target object identifier within a preset time period is collected, such as collecting its live broadcast screen (i.e., target multimedia content) within 5 minutes of determining that the live broadcast room is on, so as to determine the category of the second target object identifier according to the target multimedia content.

[0152] As a possible implementation method, it can be determined first whether the second target object identifier belongs to the object identifier in the whitelist. If not, the step of collecting the target multimedia content disseminated by the second target object identifier within a preset time period is performed to determine the category of the second target object identifier based on the target multimedia content.

[0153] Therefore, an object identifier of an abnormal category can be used to specifically identify whether other object identifiers in the same abnormal object set are spreading abnormal multimedia content, thereby not only improving the recognition speed but also improving the recognition accuracy of the object identifiers of the abnormal category.

[0154] In order to further understand the technical solution provided by the embodiments of the present application, Figure 6 The object identifier is taken as a live broadcast room identifier as an example for explanation.

[0155] See also Figure 6 , which is a schematic diagram of determining the category of an object set provided in an embodiment of the present application.

[0156] S601: If the multimedia content is transmitted through the first target object identifier, obtain a set of all abnormal objects where the first target object identifier is located.

[0157] S602: Determine whether the second target object identifier included in each abnormal object set is transmitting multimedia content. If so, execute S603; if not, end the process.

[0158] S603: Determine whether the second target object identifier included in each abnormal object set is in the white list. If so, end the process; if not, execute S604.

[0159] It can be understood that the second target object identifier in S603 is the object identifier that is transmitting the multimedia content.

[0160] S604: collecting target multimedia content disseminated by the second target object identifier within a preset time period, so as to determine the category of the second target object identifier according to the target multimedia content.

[0161] Method 2: Asynchronous recognition.

[0162] Obtain a third target object identifier that has been punished within a preset historical time period, obtain a fourth target object set in which the third target object identifier is located; and determine the category of the object identifier that has not been punished among the object identifiers included in the fourth target object set.

[0163] The embodiment of the present application does not specifically limit the length of the preset historical time period, and those skilled in the art may set it according to actual needs. For example, it may be the previous day.

[0164] As a possible implementation method, multiple third target object identifiers that have been punished within a preset historical time period can be obtained, and the secondary categories of the multiple third target object identifiers belong to the same category, so that the object identifiers of each category can be identified in a targeted manner to prevent omissions and improve the accuracy of identification.

[0165] Therefore, the object identifiers included in the fourth target object set to which the punished third target object identifier belongs are most likely also object identifiers of abnormal categories. Therefore, the categories of the object identifiers that have not been punished among the object identifiers included in the fourth target object set can be determined in a targeted manner, thereby quickly identifying the categories of the object identifiers to prevent the object identifiers of the abnormal category from not being punished.

[0166] In order to further understand the technical solution provided by the embodiments of the present application, Figure 7 The object identifier is taken as a live broadcast room identifier as an example for explanation.

[0167] See also Figure 7 , which is a schematic diagram of determining the category of an object set provided in an embodiment of the present application.

[0168] S701: Acquire multiple third target object identifiers that belong to the target category and have been punished within the previous day.

[0169] S702: Acquire multiple object sets where multiple third target object identifiers are located respectively.

[0170] S703: Determine whether each object set includes the punished object identifier, if so, execute S704; if not, execute S705.

[0171] S704: Filter the punished accounts from each object set to obtain an updated object set, and execute S705.

[0172] S705: Determine whether the object identifiers included in each object set are less than or equal to 1. If so, end the process; if not, execute S706.

[0173] S706: Remove duplicate object sets from the multiple object sets.

[0174] S707: Determine the category of each object identifier included in each object set.

[0175] Therefore, it is possible to impose a penalty on the object identifiers of the abnormal category so as to eradicate the set of objects of the abnormal category.

[0176] As a possible implementation method, the present application embodiment provides a system for determining the category of an object set, see Figure 8 , which is a schematic diagram of a system for determining the category of an object set provided by an embodiment of the present application. The system includes an object set clustering unit 801, an abnormal category identification unit 802, a list construction unit 803 and an execution unit 804. Figure 8 Each unit is described separately.

[0177] The object set clustering unit 801 is used to obtain multiple object identifiers, cluster the multiple object identifiers, and obtain multiple object sets. Figure 8 Object collection 1, object collection 2, etc. shown in .

[0178] The abnormal category identification unit 802 is used to identify an object set of abnormal category from multiple object sets based on existing categories. Figure 8 As shown, it is possible to identify whether each object set belongs to the normal category or the abnormal category based on anchor object features, live broadcast features, subject features, audience object features, violation features, environmental features, object identification features, etc.

[0179] The list construction unit 803 is used to determine the first business indicator values ​​and the second business indicator values ​​corresponding to the multiple object sets respectively, sort the first business indicator values ​​corresponding to the multiple object sets respectively to obtain a first sorting result, and sort the second indicator values ​​corresponding to the multiple object sets respectively to obtain a second sorting result, and determine the category of the object sets included in the multiple object sets according to the difference between the first sorting result and the second sorting result. Figure 8As shown, the sorting results can be based on different business indicators, such as traffic list, GMV list, reward list, member level list, violation list, etc. Thus, new categories under normal categories and new categories under abnormal categories can be mined.

[0180] The execution unit 804 is used to perform the aforementioned real-time recognition and asynchronous recognition, so as to quickly discover the live broadcast rooms of abnormal categories and eradicate the object sets of abnormal categories.

[0181] After experiments, the object set category determination system provided by the embodiment of the present application has also achieved good results in actual business scenarios. Taking the management of live pornography violations as an example, the punished anchor account (live broadcast room identifier) ​​is used as a seed to diffuse and find other accounts in the same object set, and the execution unit is sent for review and punishment. The number of accounts sent for review every day is 1,300, and the manual review violation rate is more than 86%. Combined with the real-time strategy during the event, the overall violation coverage rate has increased from 60% to 84%, and the magnitude of pornography complaints has dropped significantly. Moreover, the solution is highly reusable, has low application cost, and quickly produces results.

[0182] With respect to the method for determining the category of a set of objects described above, the present application also provides a corresponding device for determining the category of a set of objects, so that the method for determining the category of a set of objects described above can be applied and implemented in practice.

[0183] See also Fig. 9 , which is a schematic diagram of the structure of a device for determining the category of an object set provided in an embodiment of the present application. Fig. 9 As shown, the device 900 for determining the category of the object set includes: an acquisition unit 901, a clustering unit 902, a determination unit 903 and a sorting unit 904;

[0184] The acquisition unit 901 is used to acquire multiple object identifiers, where the object identifiers are carriers of multimedia content transmitted by the objects;

[0185] The clustering unit 902 is used to cluster the multiple object identifiers to obtain multiple object sets, where different object sets correspond to different categories;

[0186] The determining unit 903 is used to determine first business indicator values ​​and second business indicator values ​​corresponding to the multiple object sets respectively, where the first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to represent quantitative data of behaviors generated by the object identifiers at different times;

[0187] The sorting unit 904 is used to sort the first business indicator values ​​corresponding to the multiple object sets respectively to obtain a first sorting result, and to sort the second indicator values ​​corresponding to the multiple object sets respectively to obtain a second sorting result;

[0188] The determining unit 903 is further configured to determine categories of object sets included in the multiple object sets according to a difference between the first sorting result and the second sorting result.

[0189] It can be seen from the above technical solution that multiple object identifiers are obtained, and the object identifiers are carriers of multimedia content transmitted by the object, and the multiple object identifiers are clustered to obtain multiple object sets, so that different object sets correspond to different categories. Although the object identifiers are divided into multiple categories through the object set, it is not known what the specific content of each category is, so as to avoid limiting the recognition of newly emerging categories by classifying based on the characteristics of known categories. In order to mine the newly emerging categories to avoid the low recognition accuracy caused by bypassing and other methods, the first business indicator and the second business indicator corresponding to the multiple object sets are determined, wherein the first business indicator value and the second business indicator value are the values ​​for the same business indicator, and different business indicator values ​​are used to characterize the quantitative data of the behavior generated by the object identifier at different times. The business indicator values ​​from the same time are sorted to obtain the first sorting result and the second sorting result. Since the object identifier of the abnormal type always hopes to get a quick return, the corresponding business indicator value will fluctuate rapidly, so the difference between the first sorting result and the second sorting result can determine the category of the object set included in the multiple object sets.

[0190] Therefore, the present application no longer identifies a single object identifier, but divides multiple object identifiers into multiple categories, and performs group identification on an object set including multiple object identifiers, which can improve the identification speed. Specifically, the object identifier of each category is determined, that is, the business indicator value of the same object set at different times, and the category of the object set is determined based on the difference between the sorting results of the business indicator values. If the sorting of the business indicator values ​​of the same object set changes significantly, the category of the object set may be a newly emerged category, and this type of object set is focused on, so as to perceive the object set of abnormal categories in advance and improve the recognition accuracy of the object identifiers of abnormal categories.

[0191] As a possible implementation manner, the clustering unit 902 is specifically configured to:

[0192] Clustering the multiple object identifiers to obtain multiple first to-be-determined object sets;

[0193] Acquire business data of object identifiers included in the plurality of first pending object sets under a target dimension, where the target dimension is a normal dimension or an abnormal dimension;

[0194] Determining the plurality of object sets from the plurality of first to-be-determined object sets according to the business data;

[0195] The determining unit 903 is specifically configured to:

[0196] If the difference between the ranking of the first target object set in the first sorting result and the ranking of the first target object set in the second sorting result is greater than a first ranking threshold, and the multiple object sets are determined according to the business data under the normal dimension, the category of the first target object set is a normal category, and the first target object set is one of the multiple object sets;

[0197] If the difference between the ranking of the first target object set in the first sorting result and the ranking of the first target object set in the second sorting result is greater than a second ranking threshold, and the multiple object sets are determined based on the business data under the abnormal dimension, then the category of the first target object set is an abnormal category.

[0198] As a possible implementation manner, the determining unit 903 is specifically configured to:

[0199] Based on the business indicator and the first time, determine a first representative object identifier from each object set included in the multiple object sets, and use the business indicator value of the first representative object identifier under the business indicator as the first business indicator value;

[0200] Based on the business indicator and the second time, a second representative object identifier is determined from each object set included in the multiple object sets, and the business indicator value of the second representative object identifier under the business indicator is used as the second business indicator value. The first time and the second time are different times.

[0201] As a possible implementation manner, the clustering unit 902 is specifically configured to:

[0202] Acquire relevant data of each object identifier included in the multiple object identifiers, wherein the relevant data includes one or more combinations of behavior data of each object identifier and multimedia content transmitted through the object identifier;

[0203] Determining, based on the relevant data, an identification feature vector of each object identification;

[0204] The multiple object identifiers are clustered according to the identifier feature vectors to obtain the multiple object sets.

[0205] As a possible implementation manner, the relevant data is obtained according to multiple dimensional sub-data, and the acquisition unit 901 is further used to:

[0206] Obtaining credibility corresponding to each of the plurality of dimensional sub-data;

[0207] The relevant data is obtained from the multiple dimensional sub-data according to the credibility respectively corresponding to the multiple dimensional sub-data; wherein, the higher the credibility of the dimensional sub-data included in the relevant data, the fewer the dimensions of the dimensional sub-data obtained from the multiple dimensional sub-data.

[0208] As a possible implementation manner, the clustering unit 902 is specifically configured to:

[0209] Clustering the multiple object identifiers according to the identifier feature vectors to obtain multiple second to-be-determined object sets;

[0210] Determine the overlap between the plurality of second specific object sets, where the overlap is used to identify the overlap degree of object identifiers included in different second specific object sets;

[0211] The two second to-be-determined object sets whose overlap degree is greater than the overlap degree threshold are merged to obtain the object set.

[0212] As a possible implementation manner, the number of object identifiers included in the object set is less than an upper limit value.

[0213] As a possible implementation, the device further includes a normal category determination unit and an abnormal category determination unit;

[0214] The normal category determination unit is used to:

[0215] Acquire the credit values ​​of the object identifiers included in the second target object set, determine the average credit value corresponding to the second target object set according to the credit values, and if the average credit value is greater than a credit value threshold, determine that the primary category of the second target object set is a normal category, and the second target object set is one of the multiple object sets;

[0216] The abnormality category determination unit is used to:

[0217] Acquire abnormal labels of object identifiers included in a third target object set, determine abnormal concentration of the third target object set according to the abnormal labels, and if the abnormal concentration is greater than a concentration threshold, determine that the primary category of the third target object set is an abnormal category, acquire relevant data of the object identifiers included in the third target object set, and determine the secondary category of the third target object set according to the relevant data, wherein the secondary category is used to identify the reason why the object set belongs to the abnormal category, the third target object set is an object set among the multiple object sets, and the relevant data includes one or more combinations of behavior data of each object identifier and multimedia content transmitted through the object identifier.

[0218] As a possible implementation manner, the device further includes a real-time identification unit, which is used to:

[0219] If the multimedia content is transmitted through the first target object identifier, obtaining at least one abnormal object set where the first target object identifier is located, the category of the first target object identifier is an abnormal category, and the abnormal object set is an object set of the abnormal category;

[0220] If the second target object identifier included in the abnormal object set is transmitting multimedia content, the target multimedia content transmitted by the second target object identifier within a preset time period is collected so as to determine the category of the second target object identifier according to the target multimedia content, and the second target object identifier and the first target object identifier are different object identifiers.

[0221] As a possible implementation manner, the device further includes an asynchronous identification unit, configured to:

[0222] Obtaining the identifier of the third target object that was punished within a preset historical time period;

[0223] Acquire a fourth target object set where the third target object identifier is located;

[0224] Determine the categories of the object identifiers that have not been punished among the object identifiers included in the fourth target object set.

[0225] As a possible implementation manner, the object identifier is a live broadcast room identifier.

[0226] The present application also provides a computer device, which is the computer device described above. The computer device can be a server or a terminal device. The aforementioned object set category determination device can be built into the server or the terminal device. The computer device provided by the present application will be introduced from the perspective of hardware entity. Fig.10 The following is a schematic diagram of the server structure. Fig.11Shown is a schematic diagram of the structure of the terminal equipment.

[0227] See also Fig.10 , which is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1400 may have relatively large differences due to different configurations or performances, and may include one or more processors 1422, such as central processing units (CPU), memory 1432, one or more application programs 1442 or storage media 1430 (such as one or more massive storage devices) for data 1444. Among them, the memory 1432 and the storage medium 1430 may be temporary storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the processor 1422 may be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the server 1400.

[0228] The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0229] The steps performed by the server in the above embodiment can be based on the Fig.10 The server structure shown.

[0230] The CPU 1422 is used to perform the following steps:

[0231] Acquire multiple object identifiers, where the object identifiers are carriers of multimedia content transmitted by the objects;

[0232] Clustering the multiple object identifiers to obtain multiple object sets, where different object sets correspond to different categories;

[0233] Determine first business indicator values ​​and second business indicator values ​​respectively corresponding to the multiple object sets, where the first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to represent quantitative data of behaviors generated by the object identifiers at different times;

[0234] Sorting the first business indicator values ​​respectively corresponding to the multiple object sets to obtain a first sorting result, and sorting the second indicator values ​​respectively corresponding to the multiple object sets to obtain a second sorting result;

[0235] According to the difference between the first sorting result and the second sorting result, categories of object sets included in the multiple object sets are determined.

[0236] Optionally, the CPU 1422 may also execute method steps of any specific implementation of the method for determining the category of an object set in the embodiments of the present application.

[0237] See also Fig.11 , which is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Taking the terminal device as a smart phone as an example, Fig.11 The block diagram of the partial structure of the smart phone is shown, and the smart phone includes: a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a wireless fidelity (WiFi) module 1570, a processor 1580, and a power supply 1590. Those skilled in the art can understand that Fig.11 The structure of the smartphone shown in the figure does not constitute a limitation of the smartphone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0238] Combine the following Fig.11 A detailed introduction to the various components of a smartphone:

[0239] The RF circuit 1510 may be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information from the base station, it is sent to the processor 1580 for processing; in addition, the designed uplink data is sent to the base station.

[0240] The memory 1520 may be used to store software programs and modules. The processor 1580 implements various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 1520 .

[0241] The input unit 1530 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the smartphone. Specifically, the input unit 1530 may include a touch panel 1531 and other input devices 1532. The touch panel 1531, also known as a touch screen, can collect user touch operations on or near it and drive the corresponding connection device according to a pre-set program. In addition to the touch panel 1531, the input unit 1530 may also include other input devices 1532. Specifically, other input devices 1532 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0242] The display unit 1540 may be used to display information input by the user or information provided to the user and various menus of the smartphone. The display unit 1540 may include a display panel 1541, and the display panel 1541 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0243] The smartphone may also include at least one sensor 1550, such as a light sensor, a motion sensor, and other sensors. As for other sensors that may be configured in the smartphone, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., they will not be described in detail here.

[0244] The audio circuit 1560, the speaker 1561, and the microphone 1562 can provide an audio interface between the user and the smartphone. The audio circuit 1560 can transmit the received audio data to the speaker 1561 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1560 and converted into audio data, and then the audio data is output to the processor 1580 for processing, and then sent to another smartphone through the RF circuit 1510, or the audio data is output to the memory 1520 for further processing.

[0245] The processor 1580 is the control center of the smartphone, and uses various interfaces and lines to connect various parts of the entire smartphone, and executes various functions of the smartphone and processes data by running or executing software programs and / or modules stored in the memory 1520, and calling data stored in the memory 1520. Optionally, the processor 1580 may include one or more processing units.

[0246] The smart phone also includes a power supply 1590 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1580 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption management.

[0247] Although not shown, the smartphone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0248] In the embodiment of the present application, the memory 1520 included in the smart phone can store a computer program and transmit the computer program to the processor.

[0249] The processor 1580 included in the smart phone can execute the method for determining the category of the object set provided in the above embodiment according to the instructions in the computer program.

[0250] An embodiment of the present application further provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute the method for determining the category of a set of objects provided in the above embodiment.

[0251] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method for determining the category of the object set provided in various optional implementations of the above aspects.

[0252] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the above storage medium can be at least one of the following media: read-only memory (English: Read-Only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store computer programs.

[0253] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0254] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for determining the category of an object set, characterized in that: The method comprises: Acquire multiple object identifiers, where the object identifiers are carriers of multimedia content transmitted by the objects; Clustering the multiple object identifiers to obtain multiple object sets, where different object sets correspond to different categories; Determine first business indicator values ​​and second business indicator values ​​respectively corresponding to the multiple object sets, where the first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to represent quantitative data of behaviors generated by the object identifiers at different times; Sorting the first business indicator values ​​respectively corresponding to the multiple object sets to obtain a first sorting result, and sorting the second indicator values ​​respectively corresponding to the multiple object sets to obtain a second sorting result; According to the difference between the first sorting result and the second sorting result, categories of object sets included in the multiple object sets are determined.

2. The method according to claim 1, characterized in that The clustering of the multiple object identifiers to obtain multiple object sets includes: Clustering the multiple object identifiers to obtain multiple first to-be-determined object sets; Acquire business data of object identifiers included in the plurality of first pending object sets under a target dimension, where the target dimension is a normal dimension or an abnormal dimension; Determining the plurality of object sets from the plurality of first to-be-determined object sets according to the business data; The determining, according to the difference between the first sorting result and the second sorting result, the category of the object sets included in the multiple object sets comprises: If the difference between the ranking of the first target object set in the first sorting result and the ranking of the first target object set in the second sorting result is greater than a first ranking threshold, and the multiple object sets are determined according to the business data under the normal dimension, the category of the first target object set is a normal category, and the first target object set is one of the multiple object sets; If the difference between the ranking of the first target object set in the first sorting result and the ranking of the first target object set in the second sorting result is greater than a second ranking threshold, and the multiple object sets are determined based on the business data under the abnormal dimension, then the category of the first target object set is an abnormal category.

3. The method according to claim 1, characterized in that The determining the first business indicator values ​​and the second business indicator values ​​respectively corresponding to the plurality of object sets comprises: Based on the business indicator and the first time, determine a first representative object identifier from each object set included in the multiple object sets, and use the business indicator value of the first representative object identifier under the business indicator as the first business indicator value; Based on the business indicator and the second time, a second representative object identifier is determined from each object set included in the multiple object sets, and the business indicator value of the second representative object identifier under the business indicator is used as the second business indicator value. The first time and the second time are different times.

4. The method according to claim 1, characterized in that: The clustering of the multiple object identifiers to obtain multiple object sets includes: Acquire relevant data of each object identifier included in the multiple object identifiers, wherein the relevant data includes one or more combinations of behavior data of each object identifier and multimedia content transmitted through the object identifier; Determining, based on the relevant data, an identification feature vector of each object identification; The multiple object identifiers are clustered according to the identifier feature vectors to obtain the multiple object sets.

5. The method according to claim 4, characterized in that The relevant data is obtained based on multiple dimensional sub-data, and the method further includes: Obtaining credibility corresponding to each of the plurality of dimensional sub-data; The acquiring relevant data of each object identifier included in the multiple object identifiers includes: The relevant data is obtained from the multiple dimensional sub-data according to the credibility respectively corresponding to the multiple dimensional sub-data; wherein, the higher the credibility of the dimensional sub-data included in the relevant data, the fewer the dimensions of the dimensional sub-data obtained from the multiple dimensional sub-data.

6. The method according to claim 4, characterized in that The clustering of the multiple object identifiers according to the identifier feature vectors to obtain the multiple object sets includes: Clustering the multiple object identifiers according to the identifier feature vectors to obtain multiple second to-be-determined object sets; Determine the overlap between the plurality of second specific object sets, where the overlap is used to identify the overlap degree of object identifiers included in different second specific object sets; The two second to-be-determined object sets whose overlap degree is greater than the overlap degree threshold are merged to obtain the object set.

7. The method according to claim 6, characterized in that The number of object identifiers included in the object set is less than an upper limit value.

8. The method according to claim 1, characterized in that The method further comprises: Acquire the credit values ​​of the object identifiers included in the second target object set, determine the average credit value corresponding to the second target object set according to the credit values, and if the average credit value is greater than a credit value threshold, determine that the primary category of the second target object set is a normal category, and the second target object set is one of the multiple object sets; Acquire abnormal labels of object identifiers included in a third target object set, determine abnormal concentration of the third target object set according to the abnormal labels, and if the abnormal concentration is greater than a concentration threshold, determine that the primary category of the third target object set is an abnormal category, acquire relevant data of the object identifiers included in the third target object set, and determine the secondary category of the third target object set according to the relevant data, wherein the secondary category is used to identify the reason why the object set belongs to the abnormal category, the third target object set is an object set among the multiple object sets, and the relevant data includes one or more combinations of behavior data of each object identifier and multimedia content transmitted through the object identifier.

9. The method according to claim 1, characterized in that: The method further comprises: If the multimedia content is transmitted through the first target object identifier, obtaining at least one abnormal object set where the first target object identifier is located, the category of the first target object identifier is an abnormal category, and the abnormal object set is an object set of the abnormal category; If the second target object identifier included in the abnormal object set is transmitting multimedia content, the target multimedia content transmitted by the second target object identifier within a preset time period is collected so as to determine the category of the second target object identifier according to the target multimedia content, and the second target object identifier and the first target object identifier are different object identifiers.

10. The method according to claim 1, characterized in that The method further comprises: Obtaining the identifier of the third target object that was punished within a preset historical time period; Acquire a fourth target object set where the third target object identifier is located; Determine the categories of the object identifiers that have not been punished among the object identifiers included in the fourth target object set.

11. The method according to any one of claims 1 to 10, characterized in that: The object identifier is a live broadcast room identifier.

12. A device for determining the category of an object set, characterized in that: The device comprises: an acquisition unit, a clustering unit, a determination unit and a sorting unit; The acquisition unit is used to acquire multiple object identifiers, where the object identifiers are carriers of multimedia content transmitted by the objects; The clustering unit is used to cluster the multiple object identifiers to obtain multiple object sets, where different object sets correspond to different categories; The determining unit is used to determine first business indicator values ​​and second business indicator values ​​respectively corresponding to the multiple object sets, where the first business indicator value and the second business indicator value are numerical values ​​for the same business indicator, and different business indicator values ​​are used to represent quantitative data of behaviors generated by the object identifiers at different times; The sorting unit is used to sort the first business indicator values ​​corresponding to the multiple object sets respectively to obtain a first sorting result, and to sort the second indicator values ​​corresponding to the multiple object sets respectively to obtain a second sorting result; The determining unit is further configured to determine categories of object sets included in the multiple object sets according to a difference between the first sorting result and the second sorting result.

13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1 to 11 according to the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that When the method is executed on a computer device, the computer device is enabled to execute the method according to any one of claims 1 to 11.

Images