A data mining method, device and storage medium

Abstract

The present application relates to the field of data processing technology, and discloses a data mining method, device and storage medium, including: using a deep learning model to extract the prior features of each data to be mined, and using a pre-built a posteriori feature engineering to extract the a posteriori features of each data to be mined, and obtaining the content portrait of each data to be mined; the content portrait includes the prior features and the a posteriori features; counting all the publishing subjects of the data to be mined and the publishing relationship between each publishing subject and the data to be mined; mapping the content portraits of the data to be mined published by each publishing subject to the subject portraits of each publishing subject, and determining the target publishing subject according to the subject portraits. It is possible to realize the mining of high-quality publishing subjects on the basis of ensuring the comprehensiveness of content mining.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data mining method, device and storage medium. Background Art

[0002] In the era of big data, the base of various content data is large and the quality is uneven. High-quality content can convey information concisely, concisely and efficiently. Mining content data and content publisher data is the top priority for effectively utilizing big data information. For example, for post data, each post represents a piece of data, and the volume is often larger. It is necessary to analyze which posts are valuable to the business and which are worthless from a large number of posts. Valuable and high-quality posts can often bring unexpected information value, such as realizing the supply and distribution of high-quality content and promoting content growth. Similarly, high-quality post creators also have important information value, such as realizing the supply and distribution of high-quality creators and applying them to various business scenarios. In related technologies, there is no perfect system for effectively mining high-quality content and high-quality creators, and the mining results are often limited.

[0003] Therefore, how to solve the above technical problems urgently needs to be solved by those skilled in the art. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a data mining method, device, equipment and storage medium, which can realize the mining of high-quality publishing entities on the basis of ensuring the comprehensiveness of content mining. The specific scheme is as follows:

[0005] The first aspect of the present application provides a data mining method, comprising:

[0006] A priori features of each data to be mined are extracted using a deep learning model, and a posteriori features of each data to be mined are extracted using a pre-built posteriori features engineering to obtain a content portrait of each data to be mined; the content portrait includes the priori features and the posteriori features;

[0007] Counting all the publishing entities of the data to be mined and the publishing relationship between each publishing entity and the data to be mined;

[0008] The content portraits to be mined and published by each publishing subject are mapped to subject portraits of each publishing subject, and a target publishing subject is determined according to the subject portraits.

[0009] Optionally, the extracting prior features of each data to be mined by using a deep learning model includes:

[0010] The exquisite features of each of the data to be mined are extracted using a high-quality graphic model to obtain a first priori feature representing the exquisiteness of the data;

[0011] And / or, extracting the rarity features of each of the data to be mined using a graph classification model to obtain a second priori feature representing the rarity of the data;

[0012] And / or, utilizing an originality recognition model to extract originality features of each of the data to be mined, and obtaining a third priori feature representing the degree of originality of the data.

[0013] Optionally, the extracting the a posteriori features of each of the data to be mined by using pre-built a posteriori feature engineering includes:

[0014] Determine the basic image engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined by using the pre-built a posteriori feature engineering;

[0015] Corresponding presentation features are generated according to the basic portrait engineering features, interactive engineering features and personality-related engineering features of each post to be mined, and the posterior features representing the presentation degree of the data are obtained.

[0016] Optionally, the generating of corresponding presentation features according to the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data elements to be mined includes:

[0017] It is determined whether the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined meet the first threshold condition. If so, it is determined that the data to be mined has presentation features; if not, it is determined that the data to be mined does not have presentation features.

[0018] Optionally, mapping the post portraits of the data to be mined published by each publishing subject to subject portraits of each publishing subject includes:

[0019] The prior features and the a posteriori features in the content portraits of all the data to be mined published by each publishing entity are respectively integrated and calculated to obtain the mapped prior features and the mapped a posteriori features corresponding to each publishing entity;

[0020] The mapping prior features and the mapping posterior features are fused at the content granularity to obtain mapping fusion features, and the subject portrait of each publishing subject is obtained; the subject portrait includes the mapping prior features, the mapping posterior features and the mapping fusion features.

[0021] Optionally, determining a target publishing subject according to the subject portrait includes:

[0022] It is respectively determined whether the mapping prior features, the mapping posterior features and the mapping fusion features in the subject portrait meet the second threshold condition. If so, it is determined that the data to be mined corresponding to the subject portrait is the target publishing subject.

[0023] Optionally, after obtaining the content portrait of each of the data to be mined, the method further includes:

[0024] The gear to which the content portrait belongs is determined according to the preset portrait gear, and whether the corresponding data to be mined is the target data is judged according to the gear; the preset portrait gear is determined based on the arrangement and combination of the feature values ​​of the prior features and the a posteriori features.

[0025] Optionally, the data mining method further includes:

[0026] Obtaining seed data; the seed data is data having recall demand characteristics;

[0027] Using a data representation model to perform feature extraction on the seed data and the target data respectively, to obtain a first feature vector of the seed data and a second feature vector of the target data;

[0028] The feature similarity between the first feature vector and the second feature vector is calculated, and whether the corresponding target data is final data is determined according to the feature similarity.

[0029] A second aspect of the present application provides a data mining device, comprising:

[0030] A content portrait construction module is used to extract the prior features of each data to be mined using a deep learning model, and to extract the posterior features of each data to be mined using a pre-built posterior feature engineering, so as to obtain a content portrait of each data to be mined; the content portrait includes the prior features and the posterior features;

[0031] A statistics module, used to count all the publishing entities of the data to be mined and the publishing relationship between each publishing entity and the data to be mined;

[0032] The subject portrait construction module is used to map the content portraits to be mined and published by each publishing subject into subject portraits of each publishing subject, and determine the target publishing subject according to the subject portraits.

[0033] A third aspect of the present application provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned data mining method.

[0034] A fourth aspect of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the aforementioned data mining method is implemented.

[0035] In this application, a priori features of each data to be mined are first extracted using a deep learning model, and a pre-built a posteriori feature engineering is used to extract the a posteriori features of each data to be mined, so as to obtain the content portrait of each data to be mined; the content portrait includes the priori features and the a posteriori features; then, statistics are obtained on all the publishing subjects of the data to be mined and the publishing relationship between each publishing subject and the data to be mined; finally, the content portraits of the data to be mined published by each publishing subject are mapped to the subject portraits of each publishing subject, and the target publishing subject is determined based on the subject portraits. It can be seen that this application extracts both the priori features and the a posteriori features for the data to be mined, so that the post features reflected in the content portrait are more comprehensive. On this basis, through the mapping of the content portrait, the features at the content granularity level are mapped to the features at the publishing subject granularity level, so as to mine high-quality publishing subjects. That is, it is possible to mine high-quality publishing subjects on the basis of ensuring the comprehensiveness of content mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flow chart of a data mining method provided for this application;

[0038] Figure 2 A specific data mining method flow chart provided for this application;

[0039] Figure 3 A specific post portrait construction flow chart provided for this application;

[0040] Figure 4 A specific data mining method flow chart provided for this application;

[0041] Figure 5 A flowchart of a specific portrait mapping method provided for this application;

[0042] Figure 6 A specific data mining method flow chart provided for this application;

[0043] Figure 7 A specific high-quality rapid response mechanism flow chart provided for this application;

[0044] Figure 8 A schematic diagram of the structure of a data mining device provided in this application;

[0045] Fig. 9 A structural diagram of a data mining electronic device provided in this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The construction of high-quality content is imperative, but there is currently no complete system to effectively mine high-quality content and high-quality creators, and the mining results are often limited. In response to the above technical defects, this application provides a data mining solution that extracts both prior features and a posteriori features for the data to be mined, so that the post features reflected in the content portrait are more comprehensive. On this basis, through the mapping of the content portrait, the features at the content granularity level are mapped to the features at the publishing subject granularity level, thereby mining high-quality publishing subjects. That is, it is possible to achieve the mining of high-quality publishing subjects on the basis of ensuring the comprehensiveness of content mining.

[0048] Figure 1 A flow chart of a data mining method provided in an embodiment of the present application. Figure 1 As shown, the data mining method includes:

[0049] S11: Use a deep learning model to extract the prior features of each data to be mined, and use a pre-built a posteriori feature engineering to extract the posterior features of each data to be mined, so as to obtain a content portrait of each data to be mined; the content portrait includes the prior features and the posterior features.

[0050] In this embodiment, a priori features of each data to be mined are extracted using a deep learning model, and a pre-constructed a posteriori feature engineering is used to extract the a posteriori features of each data to be mined, so as to obtain a content portrait of each data to be mined; the content portrait includes the priori features and the a posteriori features. The data to be mined are generally articles or opinions published on online forums, such as posts. In the post mining scenario, the data to be mined are posts to be mined, and the constructed content portrait is a post portrait, which includes two parts: priori features and a posteriori features. When performing feature extraction, on the one hand, a priori features of each post to be mined are extracted using a deep learning model, and on the other hand, a posteriori features of each post to be mined are extracted using a pre-constructed a posteriori feature engineering, so as to obtain a post portrait of each post to be mined. Prior features are a kind of existing prior knowledge, which are extracted by a deep learning model and used to characterize the inherent attribute features of the post itself, that is, the result of the deep learning model reflects the inherent attributes of the post. A posteriori features are the interaction features between posts and other users, which are used to characterize the consumption characteristics of posts. The method of identifying through consumption features can be defined as a posteriori identification method. These consumption features often have time lags and are a kind of a posteriori knowledge generated by feedback outside the posts.

[0051] In this embodiment, the prior features include the exquisiteness, rarity, originality and presentation features of the data to be mined. In the post mining scenario, the prior features are reflected by the exquisiteness of the post style, the rarity of the post content and the originality of the post, and the a posteriori features are reflected by the presentation of the post in user consumption. The specific steps include the following: Figure 2 ):

[0052] S111: extracting exquisite features of each of the data to be mined using a high-quality graphic model to obtain a first priori feature representing the exquisiteness of the data.

[0053] In this embodiment, the high-quality image and text model is used to extract the exquisite features of each of the data to be mined, and the first prior feature that characterizes the exquisiteness of the data is obtained. In the post mining scenario, the high-quality image and text model is used to extract the exquisite features of each of the posts to be mined, and the first prior feature that characterizes the exquisiteness of the post is obtained. The input of the high-quality image and text model is the text and image content of the post to be mined (excluding audio, and the first 4 frames of video are extracted), and the model output is the predicted post exquisiteness score, which is a floating point number in [0,1]. A beautiful threshold is set, and the model threshold is usually selected as 0.51 (accuracy is 90%, 80%). The posts to be mined that meet the threshold conditions have exquisite features. The first prior feature corresponding to the posts to be mined with exquisite features is quantized to 1, and the first prior feature corresponding to the posts to be mined without exquisite features is quantized to 0.

[0054] S112: Utilizing the graph classification model to extract the rarity features of each of the data to be mined, and obtaining a second priori feature representing the rarity of the data.

[0055] In this embodiment, the rarity features of each of the data to be mined are extracted using a graph classification model to obtain a second prior feature that characterizes the rarity of the data. In the post mining scenario, the rarity features of each of the posts to be mined are extracted using a graph classification model to obtain a second prior feature that characterizes the rarity of the post. The input of the graph classification model is the text and image content of the post, and the model output is the predicted post classification label. The classification label is a conventional vertical classification label, including 470 categories. Of course, different rare classification categories can also be set in different scenarios. For example, posts with labels such as ancient style, motorcycle, and extreme sports are defined as posts with rarity features. The second prior feature corresponding to the posts to be mined with rarity features is quantized to 1, and the second prior feature corresponding to the posts to be mined without rarity features is quantized to 0.

[0056] S113: Utilizing the originality recognition model to extract the originality features of each of the data to be mined, and obtaining a third priori feature representing the originality of the data.

[0057] In this embodiment, the originality recognition model is used to extract the originality features of each of the data to be mined, and a third priori feature that characterizes the originality of the data is obtained. In the post mining scenario, the originality recognition model is used to extract the originality features of each of the posts to be mined, and a third priori feature that characterizes the originality of the post is obtained. The input of the originality recognition model is the text and image content of the post, and the model output is the predicted original classification label of the post. The label has two enumerations: original and non-original. Posts that hit the original label are considered to have originality features. Similarly, the third priori feature corresponding to the posts to be mined with originality features is quantized to 1, and the third priori feature corresponding to the posts to be mined without originality features is quantized to 0.

[0058] S114: Determine the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined using the pre-constructed a posteriori feature engineering.

[0059] S115: Generate corresponding presentation features according to the basic portrait engineering features, interactive engineering features and personality-related engineering features of each post to be mined, and obtain the posterior features representing the presentation degree of the data.

[0060] In this embodiment, the corresponding display features are generated according to the basic image engineering features, interactive engineering features and personality-related engineering features of each post to be mined, and the a posteriori features representing the display degree of the data are obtained. On this basis, the corresponding display features are generated according to the basic image engineering features, interactive engineering features and personality-related engineering features of each post to be mined, and the a posteriori features representing the display degree of the data are obtained. In the post mining scenario, when extracting the a posteriori features, the basic image engineering features, interactive engineering features and personality-related engineering features of each post to be mined are first determined by using the pre-constructed a posteriori feature engineering, and then the corresponding display features are generated according to the basic image engineering features, interactive engineering features and personality-related engineering features of each post to be mined, and the a posteriori features representing the display degree of the post are obtained. Specifically, it is judged whether the basic image engineering features, interactive engineering features and personality-related engineering features of each post to be mined meet the first threshold condition. If yes, it is determined that the post to be mined has the display feature, and if not, it is determined that the post to be mined does not have the display feature. That is, a corresponding recall threshold is set for each engineering feature, and the features that meet the recall conditions are considered to be engineering features that represent high quality, and the posts that meet all the recall threshold conditions are considered to have display features. Here, the posterior features corresponding to the posts to be mined with display features are still quantized to 1, and the posterior features corresponding to the posts to be mined without display features are quantized to 0.

[0061] It can be understood that the core of posterior feature extraction lies in the construction of posterior feature engineering. Posterior feature engineering defines basic portrait engineering features, interactive engineering features, and personality-related engineering features. Basic portrait engineering features can be post status, type, time, etc., interactive engineering features can be comments and likes on posts, etc., and personality-related engineering features can be the proportion of same-sex consumption and the proportion of opposite-sex consumption. These engineering features are defined because the quality of a post can be reflected through the consumption characteristics of some users. For example, the user's likes and comments on a post can reflect the popularity of the post, and the consumption characteristics can reflect the presentation of the post, thereby judging whether the post is high-quality.

[0062] The post portrait process constructed in this embodiment in the post scenario is as follows Figure 3 As shown, a post portrait is constructed by combining the two technical means of priori and a posteriori, and a high-quality post portrait system with four dimensions of exquisiteness, rarity, originality, and presentation is constructed. The high-quality capabilities of posts are reflected from multiple dimensions, reducing the limitations brought by a single priori model or one-sided a posteriori features.

[0063] S12: Count all the publishing entities of the data to be mined and the publishing relationship between each publishing entity and the data to be mined.

[0064] In this embodiment, after constructing the content portrait, in order to mine high-quality publishers, the various publishers of all the data to be mined and the publishing relationships between each publisher and the data to be mined are counted. In the post mining scenario, after constructing the post portrait, in order to mine high-quality creators, it is necessary to count the various publishers of all the posts to be mined and the publishing relationships between each publisher and the posts to be mined. The publishing relationship between the poster and the post to be mined is mainly achieved by associating the post ID and the creator ID, so that it is possible to well determine which post subject has published which posts, which is not limited in this embodiment.

[0065] S13: Mapping the content portraits to be mined and published by each publishing subject into subject portraits of each publishing subject, and determining a target publishing subject according to the subject portraits.

[0066] In this embodiment, when mining high-quality creators, the content portraits of the posts to be mined published by each of the publishing entities are mainly mapped to the subject portraits of each of the publishing entities, and the target publishing entity is determined based on the subject portraits. In the post mining scenario, the post portraits of the posts to be mined published by each of the posting entities are mainly mapped to the subject portraits of each of the posting entities, and finally the high-quality posting entities are determined based on the subject portraits. Specifically, the following steps are included: Figure 4 ):

[0067] S131: Integrate and calculate the prior features and the a posteriori features in the content portraits of all the data to be mined published by each publishing entity, and obtain the mapped prior features and the mapped a posteriori features corresponding to each publishing entity.

[0068] In this embodiment, the prior features and the a posteriori features in the content portraits of all the data to be mined published by each of the publishing entities are first integrated and calculated respectively, to obtain the mapped prior features and mapped a posteriori features corresponding to each of the publishing entities. In the post mining scenario, the prior features and the a posteriori features in the post portraits of all the posts to be mined published by each of the posting entities are first integrated and calculated respectively, to obtain the mapped prior features and mapped a posteriori features corresponding to each of the posting entities. After the post portrait is constructed, the features of the post portrait are aggregated at the user granularity. Specifically, the features of all posts under each creator are integrated and calculated, and the obtained features are mapped into high-quality creator features. That is, in the creator scenario, based on the above-mentioned post portrait, the post portrait features are mapped to the dimension of the creator, and the creator portrait (subject portrait) is constructed, thereby establishing a complete set of high-quality creator portrait system to identify high-quality creators.

[0069] For example, posting subject A publishes post 1, post 2 and post 3, and post 1, post 2 and post 3 have their own prior features and posterior features respectively. When mapping, the prior features of post 1, post 2 and post 3 are integrated and calculated to obtain the mapping prior features of posting subject A, and the posterior features of post 1, post 2 and post 3 are integrated and calculated to obtain the mapping posterior features of posting subject A. The above-mentioned integrated calculation can be to average the quantized values ​​of the features, etc., which is not limited in this embodiment.

[0070] Furthermore, in the mapping operation, time constraints can be added, that is, the features are mapped according to the attributes of the time interval to obtain the features at the creator's granularity. For example, with a one-month time window as the constraint condition, that is, the creator's prior features within one month (exquisiteness rate, rarity rate, originality rate), and the a posteriori consumption features within one month (post deletion rate, user-granularity likes, user-granularity comments, etc.), assuming that Post 1 and Post 2 are posted by Poster A within this time constraint, then only Post 1 and Post 2 need to be integrated and calculated.

[0071] S132: Perform feature fusion on the mapping prior features and the mapping posterior features at the content granularity to obtain mapping fusion features, and obtain the subject portrait of each publishing subject; the subject portrait includes the mapping prior features, the mapping posterior features and the mapping fusion features.

[0072] In this embodiment, the subject portrait includes three dimensions: mapping prior features, mapping posterior features, and mapping fusion features, so it is also necessary to obtain mapping fusion features through feature fusion. Specifically, the mapping prior features and the mapping posterior features are feature fused at the content granularity to obtain mapping fusion features, and the subject portrait of each publishing subject is obtained; the subject portrait includes the mapping prior features, the mapping posterior features, and the mapping fusion features. In the post mining scenario, the mapping prior features and the mapping posterior features are feature fused at the post granularity to obtain mapping fusion features, and the subject portrait of each posting subject is obtained. The process of constructing a subject portrait is as follows: Figure 5 shown.

[0073] S133: respectively determine whether the mapping prior features, the mapping posterior features and the mapping fusion features in the subject portrait meet the second threshold condition; if so, determine that the data to be mined corresponding to the subject portrait is the target publishing subject.

[0074] In this embodiment, the high-quality posting subject is determined mainly based on the three-dimensional features of the subject portrait. Specifically, it is judged whether the mapping prior features, the mapping posterior features and the mapping fusion features in the subject portrait meet the second threshold condition. If so, the data to be mined corresponding to the subject portrait is determined to be the target posting subject. In the post mining scenario, it is judged whether the mapping prior features, the mapping posterior features and the mapping fusion features in the subject portrait meet the second threshold condition. If so, the post to be mined corresponding to the subject portrait is determined to be a high-quality posting subject. That is, the high-quality subject identification result is output by combining the three parts of the features, and thresholds are set for different single-dimensional features respectively. The threshold setting is determined by the application scenario. The creator who meets all the threshold conditions is a high-quality posting subject.

[0075] It can be seen that the embodiment of the present application first uses a deep learning model to extract the prior features of each data to be mined, and uses the pre-constructed a posteriori feature engineering to extract the a posteriori features of each data to be mined, and obtains the content portrait of each data to be mined; the content portrait includes the prior features and the a posteriori features; then the publishing subjects of all the data to be mined and the publishing relationship between each publishing subject and the data to be mined are counted; finally, the content portraits to be mined published by each publishing subject are mapped to the subject portraits of each publishing subject, and the target publishing subject is determined according to the subject portraits. The embodiment of the present application extracts the prior features and the a posteriori features for the data to be mined at the same time, so that the post features reflected by the content portrait are more comprehensive. On this basis, through the mapping of the content portrait, the features at the content granularity level are mapped to the features at the publishing subject granularity level, thereby mining high-quality publishing subjects. That is, it is possible to mine high-quality publishing subjects on the basis of ensuring the comprehensiveness of content mining.

[0076] Figure 6 A specific data mining method flow chart provided in the embodiment of the present application. Figure 6 As shown, the data mining method includes:

[0077] S21: Use a deep learning model to extract the prior features of each data to be mined, and use a pre-built a posteriori feature engineering to extract the posterior features of each data to be mined, so as to obtain a content portrait of each data to be mined; the content portrait includes the prior features and the posterior features.

[0078] In this embodiment, regarding the specific process of the above step S21, reference may be made to the corresponding content disclosed in the above embodiments, and no further details will be given here.

[0079] S22: Determine the level to which the content portrait belongs according to the preset portrait level, and judge whether the corresponding data to be mined is target data according to the level; the preset portrait level is determined based on the arrangement and combination of the feature values ​​of the prior features and the a posteriori features.

[0080] In this embodiment, the gear of the content portrait is determined according to the preset portrait gear, and the corresponding data to be mined is determined to be the target data according to the gear; the preset portrait gear is determined based on the permutation and combination of the feature values ​​of the prior feature and the a posteriori feature. In the post mining scenario, the high-quality posting subject is mainly determined based on the two-dimensional features of the post portrait. Specifically, the gear of the post portrait is determined according to the preset portrait gear, and the corresponding post to be mined is determined to be a high-quality post according to the gear. Among them, the preset portrait gear is determined based on the permutation and combination of the feature values ​​of the prior feature and the a posteriori feature. For example, when the prior features are the three dimensions of exquisite rate, rare rate, and originality rate, and the a posteriori feature is the dimension of presentation, each dimensional feature presents a distribution of 0 or 1, and there are 24 distribution results in the four dimensions, so 24 portrait gears are preset to quantify the effect of the post portrait and mark the quality grade. In general, on the premise of having the originality feature, a post that meets any other one-dimensional feature is considered to be a high-quality post.

[0081] S23: Obtain seed data; the seed data is data having recall demand characteristics.

[0082] S24: Using a data representation model to perform feature extraction on the seed data and the target data respectively, to obtain a first feature vector of the seed data and a second feature vector of the target data.

[0083] S25: Calculate the feature similarity between the first feature vector and the second feature vector, and determine whether the corresponding target data is final data according to the feature similarity.

[0084] In this embodiment, based on the combination of both a priori and a posteriori technical means to mine high-quality content, in order to further improve the recall rate, this embodiment constructs an additional extended high-quality rapid response mechanism. For some customized business needs, customized features are extracted through a data characterization model. Specifically, seed data is first obtained; the seed data is data with recall demand characteristics; then the data characterization model is used to extract features from the seed data and the target data, respectively, to obtain a first feature vector of the seed data and a second feature vector of the target data; finally, the feature similarity of the first feature vector and the second feature vector is calculated, and based on the feature similarity, it is determined whether the corresponding target data is the final data. In the post mining scenario, customized features are extracted through the post characterization model. The post characterization model is also a deep model for customized feature extraction, which can meet the rapid recall of customized content. Specifically:

[0085] First, a seed post is obtained, and the seed post is a post with a recall demand feature. For example, if a post of an emotional category needs to be recalled, the obtained seed post is a post belonging to the emotional category. Then, the post representation model is used to extract features of the seed post and the high-quality post, respectively, to obtain the first feature vector of the seed post and the second feature vector of the high-quality post. For example, for a seed post belonging to the emotional category, the first feature vector extracted must be an emotional feature vector. By calculating the feature similarity of the first feature vector and the second feature vector, the feature similarity can be calculated by the cosine distance, which is not limited in this embodiment. Then, according to the feature similarity, it is determined whether the corresponding high-quality post is the target post. A similarity threshold is set, and the post features that meet the threshold condition are output, and the corresponding post is the post that needs to be recalled. If the similarity of the first feature vector of the seed post is higher than the similarity of the second feature vector of the high-quality post, that is, the corresponding high-quality post and the seed post are closer in category, these seed posts are determined as target posts that need to be recalled.

[0086] The above process builds a high-quality rapid response mechanism at the post granularity, which can be quickly implemented in some business scenarios with high customization requirements by directly extracting post features. The flow chart of this mechanism is as follows: Figure 7 As shown in the figure. The customized features and the features of the input posts are extracted through the post representation model, the similarity of the two feature vectors is calculated, and the result of the feature output is determined by the threshold. In addition, in the process of fast recall, the post portrait and subject portrait obtained above can be used as a supplement to achieve the tuning of the feature results.

[0087] See also Figure 8 As shown, the embodiment of the present application also discloses a data mining device, including:

[0088] The post portrait construction module 11 is used to extract the prior features of each data to be mined by using the deep learning model, and to extract the posterior features of each data to be mined by using the pre-constructed posterior feature engineering, so as to obtain the content portrait of each data to be mined; the content portrait includes the prior features and the posterior features;

[0089] A statistics module 12, used to count all the publishing entities of the data to be mined and the publishing relationship between each publishing entity and the data to be mined;

[0090] The subject portrait construction module 13 is used to map the content portraits to be mined and published by each publishing subject into subject portraits of each publishing subject, and determine the target publishing subject according to the subject portraits.

[0091] It can be seen that the embodiment of the present application first uses a deep learning model to extract the prior features of each data to be mined, and uses the pre-constructed a posteriori feature engineering to extract the a posteriori features of each data to be mined, and obtains the content portrait of each data to be mined; the content portrait includes the prior features and the a posteriori features; then the publishing subjects of all the data to be mined and the publishing relationship between each publishing subject and the data to be mined are counted; finally, the content portraits to be mined published by each publishing subject are mapped to the subject portraits of each publishing subject, and the target publishing subject is determined according to the subject portraits. The embodiment of the present application extracts the prior features and the a posteriori features for the data to be mined at the same time, so that the post features reflected by the content portrait are more comprehensive. On this basis, through the mapping of the content portrait, the features at the content granularity level are mapped to the features at the publishing subject granularity level, thereby mining high-quality publishing subjects. That is, it is possible to mine high-quality publishing subjects on the basis of ensuring the comprehensiveness of content mining.

[0092] In some specific embodiments, the content portrait construction module 11 specifically includes:

[0093] A first extraction unit is used to extract exquisite features of each of the data to be mined using a high-quality graphic model to obtain a first priori feature representing the exquisiteness of the data;

[0094] A second extraction unit is used to extract the rarity feature of each of the data to be mined by using a graph classification model to obtain a second priori feature representing the rarity of the data;

[0095] A third extraction unit is used to extract the originality features of each of the data to be mined by using the originality recognition model to obtain a third priori feature representing the originality of the data;

[0096] The fourth extraction unit is used to determine the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined by using the pre-constructed a posteriori feature engineering; and to generate corresponding presentation features according to the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the posts to be mined, so as to obtain the a posteriori features that characterize the presentation degree of the data.

[0097] In some specific embodiments, the fourth extraction unit is specifically used to determine whether the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined meet the first threshold condition. If so, it is determined that the data to be mined has presentation features; if not, it is determined that the data to be mined does not have presentation features.

[0098] In some specific embodiments, the subject portrait building module 13 specifically includes:

[0099] An integration calculation unit, used to respectively integrate and calculate the prior features and the posterior features in the content portraits of all the data to be mined published by each publishing entity, to obtain the mapped prior features and the mapped posterior features corresponding to each publishing entity;

[0100] A feature fusion unit, used for performing feature fusion on the mapping prior feature and the mapping posterior feature at the content granularity to obtain a mapping fusion feature, and obtain the subject portrait of each publishing subject; the subject portrait includes the mapping prior feature, the mapping posterior feature and the mapping fusion feature;

[0101] The judging and determining unit is used to respectively judge whether the mapping prior features, the mapping posterior features and the mapping fusion features in the subject portrait meet the second threshold condition. If so, it is determined that the data to be mined corresponding to the subject portrait is the target publishing subject.

[0102] In some specific embodiments, the data mining device further includes:

[0103] A judgment module, used to determine the gear to which the content portrait belongs according to a preset portrait gear, and judge whether the corresponding data to be mined is target data according to the gear; the preset portrait gear is determined based on the arrangement and combination of the feature values ​​of the prior feature and the a posteriori feature;

[0104] A recall module is used to obtain seed data; the seed data is data with recall demand characteristics; the seed data and target data are respectively extracted with a data characterization model to obtain a first feature vector of the seed data and a second feature vector of the target data; the feature similarity of the first feature vector and the second feature vector is calculated, and whether the corresponding target data is the final data is determined according to the feature similarity.

[0105] Furthermore, an embodiment of the present application also provides an electronic device. Fig. 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0106] Fig. 9 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data mining method disclosed in any of the aforementioned embodiments.

[0107] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0108] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0109] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the data mining method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. The data 223 can include post data collected by the electronic device 20.

[0110] Furthermore, an embodiment of the present application also discloses a storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the data mining method disclosed in any of the aforementioned embodiments are implemented.

[0111] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0112] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0113] The data mining method, device, equipment and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data mining method, characterized in that: include: A priori features of each data to be mined are extracted using a deep learning model, and a posteriori features of each data to be mined are extracted using a pre-built posteriori features engineering to obtain a content portrait of each data to be mined; the content portrait includes the priori features and the posteriori features; the data to be mined are posts to be mined; Counting all the publishing entities of the data to be mined and the publishing relationship between each publishing entity and the data to be mined; Mapping the content portraits of the data to be mined published by each publishing subject into subject portraits of each publishing subject, and determining a target publishing subject according to the subject portraits; The method of extracting the prior features of each data to be mined by using a deep learning model includes: The exquisite features of each of the data to be mined are extracted using a high-quality graphic model to obtain a first priori feature representing the exquisiteness of the data; And / or, extract the rarity features of each of the data to be mined using a graph classification model to obtain a second priori feature that characterizes the rarity of the data; set different rarity classification categories in different scenarios, and define posts with ancient style, motorcycle, and extreme sports tags as posts with rarity features; And / or, using an originality recognition model to extract the originality features of each of the data to be mined, to obtain a third priori feature representing the originality of the data; Mapping the content portrait of the to-be-mined data published by each publishing subject into a subject portrait of each publishing subject includes: The prior features and the a posteriori features in the content portraits of all the data to be mined published by each publishing entity are respectively integrated and calculated to obtain the mapped prior features and the mapped a posteriori features corresponding to each publishing entity; The mapping prior features and the mapping posterior features are fused at the content granularity to obtain mapping fusion features, and the subject portrait of each publishing subject is obtained; the subject portrait includes the mapping prior features, the mapping posterior features and the mapping fusion features; through the mapping of content portraits, the features at the content granularity level are mapped to the features at the publishing subject granularity level, and high-quality publishing subjects are mined, thereby ensuring the comprehensiveness of content mining and realizing the mining of high-quality publishing subjects.

2. The data mining method according to claim 1, characterized in that: The method of extracting the a posteriori features of each of the data to be mined by using the pre-built a posteriori feature engineering includes: Determine the basic image engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined by using the pre-constructed a posteriori feature engineering; Corresponding presentation features are generated according to the basic portrait engineering features, interactive engineering features and personality-related engineering features of each post to be mined, and the posterior features representing the presentation degree of the data are obtained.

3. The data mining method according to claim 2, characterized in that: The corresponding presentation features are generated according to the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data elements to be mined, including: It is respectively determined whether the basic portrait engineering features, interactive engineering features and personality-related engineering features of each of the data to be mined meet the first threshold condition. If so, it is determined that the data to be mined has presentation characteristics; if not, it is determined that the data to be mined does not have presentation characteristics.

4. The data mining method according to claim 1, characterized in that: Determining a target publishing subject according to the subject portrait includes: It is respectively determined whether the mapping prior features, the mapping posterior features and the mapping fusion features in the subject portrait meet the second threshold condition. If so, it is determined that the data to be mined corresponding to the subject portrait is the target publishing subject.

5. The data mining method according to any one of claims 1 to 4, characterized in that: After obtaining the content portrait of each of the data to be mined, the method further includes: The gear to which the content portrait belongs is determined according to the preset portrait gear, and whether the corresponding data to be mined is the target data is judged according to the gear; the preset portrait gear is determined based on the arrangement and combination of the feature values ​​of the prior features and the a posteriori features.

6. The data mining method according to claim 5, characterized in that: Also includes: Get seed data; The seed data is data with recall demand characteristics; Using a data representation model to perform feature extraction on the seed data and the target data respectively, to obtain a first feature vector of the seed data and a second feature vector of the target data; The feature similarity between the first feature vector and the second feature vector is calculated, and whether the corresponding target data is final data is determined according to the feature similarity.

7. A data mining device, characterized in that: include: A content portrait construction module is used to extract the prior features of each data to be mined using a deep learning model, and to extract the posterior features of each data to be mined using a pre-built posterior feature engineering, so as to obtain a content portrait of each data to be mined; the content portrait includes the prior features and the posterior features; the data to be mined is a post to be mined; A statistics module, used to count all the publishing entities of the data to be mined and the publishing relationship between each publishing entity and the data to be mined; A subject portrait construction module, used to map the content portrait of the data to be mined published by each publishing subject into a subject portrait of each publishing subject, and determine a target publishing subject according to the subject portrait; The content portrait construction module specifically includes: a first extraction unit, which is used to extract the exquisiteness features of each of the data to be mined using a high-quality graphic model to obtain a first priori feature that characterizes the exquisiteness of the data; a second extraction unit, which is used to extract the rarity features of each of the data to be mined using a graph classification model to obtain a second priori feature that characterizes the rarity of the data; different rare classification categories are set in different scenarios, and posts with ancient style, motorcycle, and extreme sports tags are defined as posts with rarity features; a third extraction unit, which is used to extract the originality features of each of the data to be mined using an originality recognition model to obtain a third priori feature that characterizes the originality of the data; The subject portrait construction module specifically includes: an integration calculation unit, which is used to integrate and calculate the prior features and the posterior features in the content portraits of all the data to be mined published by each publishing subject, so as to obtain the mapping prior features and the mapping posterior features corresponding to each publishing subject; a feature fusion unit, which is used to fuse the mapping prior features and the mapping posterior features at the content granularity, so as to obtain the mapping fusion features and the subject portrait of each publishing subject; the subject portrait includes the mapping prior features, the mapping posterior features and the mapping fusion features; through the mapping of content portraits, the features at the content granularity level are mapped to the features at the publishing subject granularity level, so as to mine high-quality publishing subjects, and realize the mining of high-quality publishing subjects on the basis of ensuring the comprehensiveness of content mining.

8. A computer-readable storage medium, characterized in that: Used to store computer executable instructions, which, when loaded and executed by a processor, implement the data mining method according to any one of claims 1 to 6.

Images