Information content quality evaluation method, device and equipment in financial and financial field

Abstract

The invention relates to an information content quality evaluation method, device and equipment in the field of finance and finance. The method comprises the following steps: extracting a content label of multimedia source information in the financial and financial field by adopting a natural language processing technology, and inputting the multimedia source information into a pre-constructed three-class information classification model to obtain a category to which each piece of information belongs; performing multi-dimensional depth feature quality evaluation and timeliness evaluation on each piece of information on the basis of content tags and surface text structure features of the information, and performing weight reduction calculation on multi-dimensional depth feature quality evaluation scores of the information according to categories to which the information belongs; and comprehensive quality evaluation of the information content is realized based on the weight-reduced multi-dimensional depth feature quality evaluation score and the timeliness evaluation score. According to the method, comprehensive, real-time and multi-dimensional information content quality evaluation service can be provided for financial institutions, and investors are assisted in information screening and investment decision making.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method, device and equipment for evaluating the quality of information content in the financial and economic field. Background Art

[0002] By conducting quality assessments on information content in the financial and economic fields, financial operators can be assisted in making personalized recommendations and precise information push, thereby promoting user activity.

[0003] However, existing information content quality assessment methods mostly rely on surface features, such as article layout, number of keywords, number of paragraphs, etc. These methods cannot deeply analyze the content quality of information articles and lack quality assessment of deep-level content features, such as topicality, attractiveness, credibility, etc., resulting in the inability to reflect market hot spots and information timeliness in real time. In addition, existing information content quality assessment methods often lack flexibility and adaptability, and are difficult to cope with diverse information content. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device and equipment for evaluating the quality of information content in the financial and economic fields in response to the above-mentioned technical problems, so as to achieve comprehensive, real-time and multi-dimensional quality evaluation, accurately judge the quality of financial information content in the financial vertical field, and create a healthy and high-quality content ecology.

[0005] A method for evaluating the quality of information content in the field of finance and economics, the method comprising: Natural language processing technology is used to extract content labels of multimedia source information in the financial and economic fields, and the category of each piece of information is obtained by inputting the multimedia source information into the pre-built financial / non-financial information classification model, the advertising soft text information classification model and the standardized information classification model; among them, the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information and general information; Based on the content labels and surface text structure characteristics of the information, a multi-dimensional deep feature quality evaluation and timeliness evaluation are conducted on each piece of information, and the multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs. Based on the downgraded multi-dimensional deep feature quality evaluation score and timeliness evaluation score, a comprehensive quality evaluation of the information content is achieved; among which, the multi-dimensional deep feature quality evaluation includes quality evaluation of topicality, attractiveness, richness, opinion, credibility, emotionality and fluency.

[0006] In one embodiment, the quality assessment of topicality specifically includes: Retrieve the topic tags in the content tags of the information, and assign corresponding weights according to the importance of each type of topic tag. The weight value is Among them, content tags include individual stock tags, industry tags, sector tags, fund tags and bond tags, etc.; topic tags include macro policy tags, real-time hot sector tags, stage hot sector tags, leading individual stock tags, important company event tags and information hot sector tags; When the weight of any type of topical label among macro policy labels, real-time hot sector labels, stage hot sector labels, leading stock labels, and important company event labels reaches a preset threshold, the topical label of this type is matched with the topical label vocabulary of the corresponding category that is pre-maintained and constructed, and the topical matching score of the topical label of this type and the matching item is calculated; wherein, if there are multiple matching items, the highest score is obtained as the topical matching score of the topical label of this type; wherein, the topical label vocabulary includes the macro policy label vocabulary, the real-time hot sector label vocabulary, the stage hot sector label vocabulary, the leading stock label vocabulary, and the important company event label vocabulary; The initial topic score of the information is obtained by weighted calculation of the topic matching scores of macro policy tags, real-time hot sector tags, stage hot sector tags, leading stock tags, important company event tags, information hot sector tag scores, tag number scores, and common tag scores, which is expressed as Initial topic score = macro policy label matching score × a1 + real-time hot sector label matching score × a2 + stage hot sector label matching score × a3 + leading stock label matching score × a4 + important company event label matching score × a5 + information hot sector label score × a6 + label number score × a7 + general label score × a8; in, is the topic weighting coefficient, and its value is determined by the time period of information release; the score of the hot information section label is determined by the frequency of occurrence of the hot information section label in the information; the label number score is determined by the number of content labels contained in the information; the common label score is determined by the weight of the content labels contained in the information; The topic score of the information is calculated based on the initial topic score and the fixed bonus of the information, and is expressed as Topic score = 100 / (1+EXP(2-0.017×initial topic score))+fixed bonus points; Among them, EXP represents the natural exponential function; fixed bonus is used as a complementary strategy for topic quality evaluation, and its value is determined by the pre-written topic condition rules. When the information content hits the topic condition rules, a fixed bonus value is assigned; otherwise, no fixed bonus value is assigned.

[0007] In one embodiment, the quality assessment of attractiveness specifically includes: Retrieve the title keywords of the news, match the title keywords with the pre-maintained and constructed title eye-catching word library and clickbait word library, and sum the calculated title eye-catching word matching score and clickbait matching score to get the news attractiveness score, which is expressed as Attractiveness score = title eye-catching words matching score + clickbait matching score.

[0008] In one embodiment, the quality assessment of richness specifically includes: Based on the surface text structure characteristics of the information, the quality of the information title word count, paragraph number, paragraph length, sentence length, number of numbers, paragraph boldness, number of pictures and number of words is evaluated. The information richness score is calculated by comprehensively calculating the information title word count score, paragraph number score, paragraph length score, sentence length score, number of numbers score, paragraph boldness score, number of pictures score and number of words score, which is expressed as Richness score = ((title word count score + paragraph number score + paragraph length score + sentence length score + number ratio score + paragraph bold score + picture number score) / 7×0.4+0.6×text word count score).

[0009] In one embodiment, the quality evaluation of opinion, credibility, emotion and fluency specifically includes: Retrieve the opinion expression words mentioned in the title and text of the news, match the opinion expression words with the pre-maintained opinion keyword word library and opinion regular table, and sum the calculated opinion keyword matching score and opinion regular matching score to obtain the opinion score of the news, which is expressed as Opinion score = opinion keyword matching score + opinion regular matching score; Retrieve the credibility expression words mentioned in the text of the news and obtain the media source of the news, match the credibility expression words with the credibility keyword word library constructed in advance, and calculate the credibility keyword matching score, match the media source of the news with the media source library constructed in advance, and calculate the media source matching score, perform weighted summation of the credibility keyword matching score and the media source matching score, and obtain the credibility score of the news, which is expressed as Credibility score = media source matching score × 0.7 + credibility keyword matching score × 0.3; Retrieve the emotional expressions mentioned in the text and title of the news, match the emotional expressions in the text and title with the emotional word database maintained and constructed in advance, and perform weighted summation of the calculated text emotional word matching score and title emotional word matching score to obtain the emotional score of the news, which is expressed as Sentiment score = text sentiment word matching score × 0.7 + title sentiment word matching score × 0.3; The information is perplexed and calculated based on the pre-built content fluency model to obtain the information fluency score. The content fluency model is trained using the NN language model.

[0010] In one embodiment, the timeliness evaluation specifically includes: Extract the content tags of the information and assign corresponding weights to each type of content tag according to its importance. When the weight of a type of content tag reaches a certain threshold, match the content tag with the pre-maintained and constructed tag timeliness vocabulary, and calculate the timeliness matching score between the content tag and the matching item. If there are multiple matching items, the highest score is taken as the timeliness matching score of the content tag. Retrieve the timeliness keywords mentioned in the news title and text, match the timeliness keywords with the pre-maintained and constructed timeliness keyword thesaurus, and calculate the title timeliness matching score of the title timeliness keywords and the matching items, and the text timeliness matching score of the text timeliness keywords and the matching items; if there are multiple matching items, add up the multiple title or text timeliness matching scores to obtain the final title timeliness matching score and text timeliness matching score; The timeliness evaluation score of the information is calculated by combining the timeliness matching score of the content tag, the timeliness matching score of the title, and the timeliness matching score of the text, which is expressed as Timeliness evaluation score = 100 / (1+EXP(2.2-0.025*(timeliness matching score of content tag + timeliness matching score of title + timeliness matching score of text))); Wherein, EXP represents the natural exponential function.

[0011] In one embodiment, the multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs, and the downgraded multi-dimensional deep feature quality evaluation score is obtained, which is expressed as The multi-dimensional deep feature quality evaluation score after downgrading = information downgrading coefficient × (topic score × topic coefficient + attractiveness score × attractiveness coefficient + credibility score × credibility coefficient + opinion score × opinion coefficient + fluency score × fluency coefficient + emotion score × emotion coefficient + richness score × richness coefficient); The value of the information weight reduction coefficient is determined by the category to which the information belongs and the manually maintained weight reduction regular rules. When the category to which the information belongs matches any of non-financial information, advertising soft article information, standardized information, and announcement information, or when the title or text of the information matches any of the weight reduction regular rules, the information weight reduction coefficient is assigned a value; if multiple assignment conditions are met at the same time, the minimum value is taken as the final value of the information weight reduction coefficient; Among them, the values ​​of topic coefficient, attractiveness coefficient, richness coefficient, opinion coefficient, credibility coefficient, emotion coefficient and fluency coefficient are determined by the number of words in the information body and the time period when the information is released.

[0012] A device for evaluating the quality of information content in the field of finance and economics, the device comprising: The content label extraction and information classification module is used to extract content labels of multimedia source information in the financial and economic fields by using natural language processing technology, and obtain the category of each piece of information by inputting the multimedia source information into the pre-built financial / non-financial information classification model, the advertising soft text information classification model and the standardized information classification model; among which, the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information and ordinary information; The quality assessment module is used to conduct multi-dimensional deep feature quality assessment and timeliness evaluation on each piece of information based on the content labels and surface text structure features of the information, and to downgrade the multi-dimensional deep feature quality assessment score of the information according to the category to which the information belongs. Based on the downgraded multi-dimensional deep feature quality assessment score and timeliness evaluation score, a comprehensive quality assessment of the information content is achieved; among which, the multi-dimensional deep feature quality assessment includes quality assessment of topicality, attractiveness, richness, opinion, credibility, emotion and fluency.

[0013] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Natural language processing technology is used to extract content labels of multimedia source information in the financial and economic fields, and the category of each piece of information is obtained by inputting the multimedia source information into the pre-built financial / non-financial information classification model, the advertising soft text information classification model and the standardized information classification model; among them, the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information and general information; Based on the content labels and surface text structure characteristics of the information, a multi-dimensional deep feature quality evaluation and timeliness evaluation are conducted on each piece of information, and the multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs. Based on the downgraded multi-dimensional deep feature quality evaluation score and timeliness evaluation score, a comprehensive quality evaluation of the information content is achieved; among which, the multi-dimensional deep feature quality evaluation includes quality evaluation of topicality, attractiveness, richness, opinion, credibility, emotionality and fluency.

[0014] The above-mentioned information content quality assessment method, device and equipment in the financial and economic field, by using natural language processing technology and information classification models to extract information content labels and classify information, can perform multi-dimensional deep feature quality assessment and timeliness evaluation for different types of information, thereby improving the depth and breadth of information quality assessment, and based on the topic quality assessment and timeliness evaluation in the multi-dimensional deep feature quality assessment, it can keep up with the hot spots of the financial market and improve the real-time and accuracy of information quality assessment. In addition, this application can also adjust the content labeling system and evaluation dimensions according to the characteristics of different financial fields, adapt to information of different types and fields, and improve the flexibility of information quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a method for evaluating the quality of information content in the financial and economic field in one embodiment; Figure 2 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0017] In one embodiment, Figure 1 As shown, a method for evaluating the quality of information content in the financial and economic field is provided, comprising the following steps: Step S1, using natural language processing technology to extract content tags of multimedia source information in the financial and economic fields, and obtaining the category of each piece of information by inputting the multimedia source information into a pre-built financial / non-financial information classification model, an advertising and soft text information classification model, and a standardized information classification model; wherein the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising and soft text information classification model labels the multimedia source information as advertising and soft text information and non-advertising and soft text information; and the standardized information classification model labels the multimedia source information as standardized information, announcement information, and general information.

[0018] Specifically, step S1 includes the following steps: First, we use natural language processing (NLP) technologies such as word segmentation algorithms, entity disambiguation algorithms, institutional viewpoint disambiguation algorithms, and label weight algorithms to extract content labels of multimedia source information in the financial and economic fields, including individual stock labels, industry labels, sector labels, fund labels, bond labels, etc.

[0019] Then, the category to which each piece of information belongs is obtained by inputting the multimedia source information into the pre-built financial / non-financial information classification model, the advertising soft article information classification model and the standardized information classification model.

[0020] Among them, the financial / non-financial information classification model labels multimedia source information as financial information and non-financial information. Specifically, the financial / non-financial information classification model uses the BERT model for training, and its training goal is an F1 score of 0.97.

[0021] The advertising soft article information classification model marks multimedia source information as advertising soft article information and non-advertising soft article information. Specifically, the advertising soft article information classification model uses the fasttext model for training, and its training target is an F1 score of 0.90.

[0022] Fasttext is a fast text classifier that embeds word vectors and N-gram vectors into a shared vocabulary and uses a single-layer linear model for classification.

[0023] The standardized information classification model marks multimedia source information as standardized information, announcement information, and general information. Specifically, the standardized information classification model uses the xgboost model for training, and its training target is an F1 score of 0.98. Among them, xgboost is an ensemble learning algorithm, which is an improvement of the gradient boosting tree. It combines multiple weak learners (usually decision trees) to build a powerful ensemble model.

[0024] Step S2, based on the content labels and surface text structure features of the information, conduct multi-dimensional deep feature quality assessment and timeliness evaluation on each piece of information, and downgrade the multi-dimensional deep feature quality assessment score of the information according to the category to which the information belongs, and implement a comprehensive quality assessment of the information content based on the downgraded multi-dimensional deep feature quality assessment score and timeliness evaluation score; wherein, the multi-dimensional deep feature quality assessment includes quality assessment of topicality, attractiveness, richness, opinion, credibility, emotion, and fluency.

[0025] Specifically, the quality evaluations of topicality, attractiveness, richness, opinion, credibility, emotion, and fluency in step S2 are: (1) Topic Degree. Topic degree refers to the degree to which the topic words such as market, company, industry, etc. mentioned in the news are of great significance to stock investment. Topic degree reflects the popularity of the content, which refers to the degree to which the content of the news is concerned by the market, such as the interpretation of macroeconomic policies that affect the market, the analysis of industry sectors with high stock price increases on the day, and the analysis of recent hot stocks and industry sectors. The higher the topic degree score, the greater the impact of the content mentioned in the news on the market.

[0026] The quality assessment of topicality specifically includes: Retrieve the topic tags in the content tags of the information, and assign corresponding weights according to the importance of each type of topic tag. The weight value is Among them, topic tags include macro policy tags, real-time hot sector tags, stage hot sector tags, leading stock tags, important company event tags and information hot sector tags; among them, the data of real-time hot sector tags, leading stock tags and information hot sector tags are updated every day according to market changes.

[0027] When the weight of any type of topical label among macro policy labels, real-time hot sector labels, stage hot sector labels, leading stock labels, and important company event labels reaches a preset threshold, this type of topical label is matched with the topical label vocabulary of the corresponding category that is pre-maintained and constructed, and the topical matching score of this type of topical label and the matching item is calculated; among which, if there are multiple matching items, the highest score is taken as the topical matching score of this type of topical label; among which, the topical label vocabulary includes the macro policy label vocabulary, the real-time hot sector label vocabulary, the stage hot sector label vocabulary, the leading stock label vocabulary, and the important company event label vocabulary.

[0028] The initial topic score of the information is obtained by weighted calculation of the topic matching scores of macro policy tags, real-time hot sector tags, stage hot sector tags, leading stock tags, important company event tags, information hot sector tag scores, tag number scores, and common tag scores, which is expressed as Initial topic score = macro policy label matching score × a1 + real-time hot sector label matching score × a2 + stage hot sector label matching score × a3 + leading stock label matching score × a4 + important company event label matching score × a5 + information hot sector label score × a6 + label number score × a7 + ordinary label score × a8.

[0029] in, is the topic weighting coefficient, and its value is determined by the time period of information release. Specifically, the value of the topic weighting coefficient is shown in Table 1, and the time period distribution of information release is shown in Table 2.

[0030] Table 1 Values ​​of topic weighting coefficients

[0031] Table 2 Time period distribution of information release

[0032] As can be seen from Table 1, generally macroeconomic policies are released before and after the market closes and are relatively important, so the corresponding weights are relatively high.

[0033] The score of the information hot section label is determined by the frequency of occurrence of the information hot section label in the information. Specifically, the calculation of the information hot section label score includes: taking the information of the multimedia source of the day, counting the frequency of occurrence of the information hot section labels such as Shanghai and Shenzhen A-shares, industries, and sectors with a weight greater than 70 in these information, taking the labels with a frequency greater than 6, arranging them in descending order according to the frequency, and assigning a certain score to each interval, that is, the top 5 are assigned 100 points, the top 6 to 20 are assigned 70 points, and the top 21 to 30 are assigned 50 points.

[0034] The tag count score is determined by the number of content tags contained in the news. Specifically, the calculation of the tag count score includes: calculating the number of content tags contained in the news, and assigning a certain score according to the number of content tags, that is, if the number of content tags is greater than or equal to 30, it is assigned 100 points, if the number of content tags is greater than or equal to 15 and less than 30, it is assigned 50 points, if the number of content tags is greater than or equal to 4 and less than 15, it is assigned 30 points, and if the number of content tags is less than 4, it is assigned 0 points. The purpose of calculating the tag count score is to distinguish the topic mentioned in long news and short news.

[0035] The general tag score is determined by the weight of the content tags contained in the information. Specifically, the calculation of the general tag score includes: taking the content tags contained in the article with a weight greater than or equal to 70 and containing individual stocks, sectors, and industry tags, assigning 100 points; if the content tag weight is greater than or equal to 70 and does not contain individual stocks, sectors, and industry tags, assigning 60 points; otherwise, assigning 0 points. The general tag score gives a certain score to information with financial topics. This strategy is a bottom-line strategy.

[0036] The topic score of the information is calculated based on the initial topic score and the fixed bonus of the information, and is expressed as Topic score = 100 / (1+EXP(2-0.017×initial topic score))+fixed bonus points; Wherein, EXP represents the natural exponential function.

[0037] As a complementary strategy for topic quality assessment, fixed bonus points are determined by pre-written topic condition rules. When the information content hits the topic condition rules, fixed bonus points are given; otherwise, no fixed bonus points are given. Among them, topic condition rules are specific condition rules written based on the characteristics of media sources and the experience of business operation experts for the title, text, media source, release time and other dimensions of certain information.

[0038] (2) Appeal. Appeal refers to the degree to which a news title appeals to readers. An attractive title can attract more readers to click on and read the news, thereby increasing the exposure and dissemination effect of the news.

[0039] The quality assessment of attractiveness specifically includes: Retrieve the title keywords of the news, match the title keywords with the pre-maintained and constructed title eye-catching word library and clickbait word library, and sum the calculated title eye-catching word matching score and clickbait matching score to get the news attractiveness score, which is expressed as Attractiveness score = title eye-catching word matching score + clickbait matching score. The attractiveness score ranges from 0 to 100.

[0040] (3) Richness. Richness reflects that the reporting of news-related information is relatively comprehensive, detailed, and in-depth, and can provide more references and decision-making basis.

[0041] The quality assessment of richness specifically includes: Based on the surface text structure characteristics of the information, the quality of the information title word count, paragraph number, paragraph length, sentence length, number of numbers, paragraph boldness, number of pictures and number of words is evaluated. The information richness score is calculated by comprehensively calculating the information title word count score, paragraph number score, paragraph length score, sentence length score, number of numbers score, paragraph boldness score, number of pictures score and number of words score, which is expressed as Richness score = ((title word count score + paragraph number score + paragraph length score + sentence length score + number ratio score + paragraph bold score + picture number score) / 7×0.4+0.6×text word count score).

[0042] Among them, the title word count score is obtained after removing the tag symbols (such as commas, periods, question marks, etc.) of the information title. If the title word count is less than 20 words or greater than 80 words, the title word count score is 20 points; in other cases, the title word count score is 100 points.

[0043] The paragraph number score is calculated by counting the number of paragraphs in the HTML (Hypertext Markup Language) text of the main body of the information. If the number of paragraphs is greater than or equal to 10 and less than 40, it is assigned 100 points; if the number of paragraphs is greater than or equal to 3 and less than 9, it is assigned 80 points, and 0 points in other cases.

[0044] The paragraph length score is the number of words in the news text after removing the HTML text divided by the number of paragraphs. When the paragraph length is greater than or equal to 30 and less than 200, it is assigned 100 points; when the paragraph length is greater than or equal to 201 and less than or equal to 400, it is assigned 60 points; otherwise, it is assigned 20 points.

[0045] The sentence length score is calculated by removing the HTML text from the main body of the news, using the period, question mark, and exclamation mark as the demarcation criteria, and counting the average length of the sentences. If the sentence length is greater than or equal to 30 and less than 100, it is assigned 100 points; if the sentence length is greater than or equal to 101 and less than 200, it is assigned 60 points; otherwise, it is assigned 40 points.

[0046] The digital proportion score is the ratio of the number of 0-9 numbers in the text to the number of words in the text after removing the HTML text. When the digital proportion is greater than 0.3, it is 0 points. In order to distinguish pure data information. When the digital proportion is greater than or equal to 0.01 and less than 0.3, it is 100 points; otherwise, it is 0 points.

[0047] The paragraph bold score is the number of subheadings in the main text of the information. The subheadings are identified as <strong>When the number of bold paragraphs is greater than or equal to 4 and less than 7, the value is 100 points; when the number of bold paragraphs is greater than or equal to 8 and less than 15, the value is 60 points; when the number of bold paragraphs is greater than or equal to 2 and less than 3, the value is 40 points; otherwise, the value is 0 points.

[0048] The picture count score is the number of pictures in the text. When the number of pictures is greater than or equal to 3 and less than 7, it is assigned 100 points; when the number of pictures is greater than or equal to 8 and less than 12, it is assigned 60 points; when the number of pictures is greater than or equal to 1 and less than 2, it is assigned 40 points; otherwise, it is 0 points.

[0049] The word count score is calculated by removing the HTML text from the body of the information. Most financial information is short text when released during intraday trading hours, and long text analysis when released after hours. Therefore, the word count score needs to be calculated based on the release period of the information.

[0050] The word count score assignment is shown in Table 3.

[0051] Table 3 Score assignment for the number of words in the text

[0052] (4) Opinion. Opinion refers to the expression of one's own attitude, view or opinion based on objective facts and evidence when delivering news or information. It includes the opinions expressed by analysts, such as: XX Securities analyst said, and forward-looking opinions on a certain policy or time, such as: future, possible, and expected. The quality assessment of opinion includes: Retrieve the opinion expression words mentioned in the title and text of the news, match the opinion expression words with the pre-maintained opinion keyword word library and opinion regular table, and sum the calculated opinion keyword matching score and opinion regular matching score to obtain the opinion score of the news, which is expressed as Opinion score = opinion keyword matching score + opinion regular matching score. The full score of opinion score is 100.

[0053] (5) Credibility. Credibility refers to the authenticity, reliability and authority of the information. Credibility is particularly important for financial information, because the authenticity and accuracy of financial information are directly related to the correctness of investment decisions and risk control. The quality assessment of credibility includes: Retrieve the credibility expression words mentioned in the text of the news and obtain the media source of the news, match the credibility expression words with the credibility keyword word library constructed in advance, and calculate the credibility keyword matching score, match the media source of the news with the media source library constructed in advance, and calculate the media source matching score, perform weighted summation of the credibility keyword matching score and the media source matching score, and obtain the credibility score of the news, which is expressed as Credibility score = media source matching score × 0.7 + credibility keyword matching score × 0.3.

[0054] (6) Sentiment. Sentiment refers to the emotion or emotional state conveyed by the information. In the field of finance and economics, sentiment usually refers to market sentiment, that is, the emotional state of market participants towards a certain asset or market. Market sentiment usually includes different emotional states such as optimism, pessimism, and panic. The quality assessment of sentiment includes: Retrieve the emotional expressions mentioned in the text and title of the news, match the emotional expressions in the text and title with the pre-maintained emotional word database (including positive words, negative words, degree adverbs, etc.), and perform weighted summation of the calculated text emotional word matching score and title emotional word matching score to obtain the emotional score of the news, expressed as Sentiment score = text sentiment word matching score × 0.7 + title sentiment word matching score × 0.3.

[0055] (7) Fluency. Fluency refers to the difficulty of understanding the content of information, which is mainly reflected by perplexity. Perplexity is an indicator used to evaluate the performance of a language model. It measures the degree to which the model understands the complexity of a sentence. The smaller the perplexity, the stronger the model's ability to predict the sentence, that is, the more accurate the model's understanding of the sentence. Conversely, it means that the model has difficulty understanding the sentence. The quality assessment of fluency specifically includes: The information is perplexed and calculated based on the pre-built content fluency model to obtain the information fluency score. The content fluency model is trained using the NN language model.

[0056] Specifically, the timeliness evaluation in step S2 refers to the duration of information delivery, including short-term, medium-term and long-term.

[0057] Short term: market market reports, morning reports, morning reviews, etc. related to intraday trading trends.

[0058] Mid-term: market review and comments after the stock market closes, analysis of hot sectors, industry analysis, macro policies, interpretation of industry policies, trends of institutions and large market funds, daily reports, weekly reports, company announcements (stock increase and decrease, share repurchase, etc.), and company performance.

[0059] Long-term: investment strategies, securities market investment education, etc.

[0060] The timeliness evaluation specifically includes: Extract the content tags of the information and assign corresponding weights to each type of content tag according to its importance. When the weight of a type of content tag reaches a certain threshold, match the content tag with the pre-maintained and constructed tag timeliness vocabulary, and calculate the timeliness matching score between the content tag and the matching item. If there are multiple matching items, the highest score is taken as the timeliness matching score of the content tag. Retrieve the timeliness keywords mentioned in the news title and text, match the timeliness keywords with the pre-maintained and constructed timeliness keyword thesaurus, and calculate the title timeliness matching score of the title timeliness keywords and the matching items, and the text timeliness matching score of the text timeliness keywords and the matching items; if there are multiple matching items, add up the multiple title or text timeliness matching scores to obtain the final title timeliness matching score and text timeliness matching score; The timeliness evaluation score of the information is calculated by combining the timeliness matching score of the content tag, the timeliness matching score of the title, and the timeliness matching score of the text, which is expressed as Timeliness evaluation score = 100 / (1+EXP(2.2-0.025*(timeliness matching score of content tag + timeliness matching score of title + timeliness matching score of text))); Wherein, EXP represents the natural exponential function.

[0061] Furthermore, in step S2, the multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs, and the downgraded multi-dimensional deep feature quality evaluation score is obtained, which is expressed as The multi-dimensional deep feature quality assessment score after downgrading = information downgrading coefficient × (topic score × topic coefficient + attractiveness score × attractiveness coefficient + credibility score × credibility coefficient + opinion score × opinion coefficient + fluency score × fluency coefficient + emotion score × emotion coefficient + richness score × richness coefficient).

[0062] The maximum score for the quality assessment of each dimension is 100 points.

[0063] Among them, the value of the information weight reduction coefficient is determined by the category to which the information belongs and the manually maintained regular rules for weight reduction. When the category to which the information belongs hits any of non-financial information, advertising information, standardized information, and announcement information, or when the title or text of the information hits any regular rule for weight reduction, the information weight reduction coefficient is assigned a value; if multiple assignment conditions are met at the same time, the minimum value is taken as the final value of the information weight reduction coefficient.

[0064] Specifically, when the information is non-financial information, the information reduction coefficient is 0.3; when the information is advertising information, the information reduction coefficient is 0.41; when the information is standardized information or announcement information, the information reduction coefficient is 0.8; when the information hits any manually maintained regular reduction rule, the information reduction coefficient is 0.63.

[0065] Among them, the values ​​of topic coefficient, attractiveness coefficient, richness coefficient, opinion coefficient, credibility coefficient, emotion coefficient and fluency coefficient are determined by the number of words in the information body and the time period of information release. Specifically, when the number of words in the information body is greater than or equal to 250 words, the coefficient values ​​of each dimension are shown in Table 4.

[0066] Table 4 Coefficient values ​​of each dimension in different time periods when the number of words in the news is greater than or equal to 250

[0067] Specifically, when the number of words in the information text is less than 250, the coefficient values ​​of each dimension are as shown in Table 5.

[0068] Table 5 The coefficient values ​​of each dimension in different time periods when the number of words in the news text is less than 250

[0069] In summary, this application uses natural language processing technology and information classification models to extract information content labels and classify information, and can perform multi-dimensional deep feature quality evaluation and timeliness evaluation for different types of information, thereby improving the depth and breadth of information quality evaluation. In addition, based on the topic quality evaluation and timeliness evaluation in the multi-dimensional deep feature quality evaluation, it can keep up with the hot spots in the financial market and improve the real-time and accuracy of information quality evaluation. In addition, this application can also adjust the content labeling system and evaluation dimensions according to the characteristics of different financial fields, adapt to information of different types and fields, and improve the flexibility of information quality evaluation. And the information content quality evaluation method in the financial and economic fields provided by this application has been verified through experiments, simulations and actual use. Experimental results show that this scheme can effectively improve the accuracy of information content quality evaluation in the financial and economic fields, with an accuracy rate of 90%.

[0070] In one embodiment, a device for evaluating the quality of information content in the financial and economic field is provided, comprising: The content label extraction and information classification module is used to extract content labels of multimedia source information in the financial and economic fields by using natural language processing technology, and obtain the category of each piece of information by inputting the multimedia source information into the pre-built financial / non-financial information classification model, the advertising soft text information classification model and the standardized information classification model; among which, the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information and ordinary information; The quality assessment module is used to conduct multi-dimensional deep feature quality assessment and timeliness evaluation on each piece of information based on the content labels and surface text structure features of the information, and to downgrade the multi-dimensional deep feature quality assessment score of the information according to the category to which the information belongs. Based on the downgraded multi-dimensional deep feature quality assessment score and timeliness evaluation score, a comprehensive quality assessment of the information content is achieved; among which, the multi-dimensional deep feature quality assessment includes quality assessment of topicality, attractiveness, richness, opinion, credibility, emotion and fluency.

[0071] The specific definition of the information content quality assessment device in the financial and economic field can be found in the above-mentioned definition of the information content quality assessment method in the financial and economic field, which will not be repeated here. Each module in the above-mentioned information content quality assessment device in the financial and economic field can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0072] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for evaluating the quality of information content in the financial and economic fields is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0073] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0074] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Natural language processing technology is used to extract content labels of multimedia source information in the financial and economic fields, and the category of each piece of information is obtained by inputting the multimedia source information into the pre-built financial / non-financial information classification model, the advertising soft text information classification model and the standardized information classification model; among them, the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information and general information; Based on the content labels and surface text structure characteristics of the information, a multi-dimensional deep feature quality evaluation and timeliness evaluation are conducted on each piece of information, and the multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs. Based on the downgraded multi-dimensional deep feature quality evaluation score and timeliness evaluation score, a comprehensive quality evaluation of the information content is achieved; among which, the multi-dimensional deep feature quality evaluation includes quality evaluation of topicality, attractiveness, richness, opinion, credibility, emotionality and fluency.

[0075] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0076] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.< / strong>

Claims

1. A method for evaluating the quality of information content in the financial and economic field, characterized in that: The method comprises: Natural language processing technology is used to extract content tags of multimedia source information in the financial and economic fields, and the category of each piece of information is obtained by inputting the multimedia source information into a pre-built financial / non-financial information classification model, an advertising soft text information classification model, and a standardized information classification model; wherein the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information, and general information; Based on the content labels and surface text structure characteristics of the information, a multi-dimensional deep feature quality evaluation and timeliness evaluation are performed on each piece of information, and the multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs. Based on the downgraded multi-dimensional deep feature quality evaluation score and timeliness evaluation score, a comprehensive quality evaluation of the information content is achieved; wherein, the multi-dimensional deep feature quality evaluation includes quality evaluation of topicality, attractiveness, richness, opinion, credibility, emotion and fluency.

2. The method according to claim 1, characterized in that The quality assessment of topicality specifically includes: Retrieve the topic tags in the content tags of the information, and assign corresponding weights according to the importance of each type of topic tag. The weight value is ; Among them, the content tags include individual stock tags, industry tags, sector tags, fund tags and bond tags; the topic tags include macro policy tags, real-time hot sector tags, stage hot sector tags, leading individual stock tags, important company event tags and information hot sector tags; When the weight of any type of topical label among macro policy labels, real-time hot sector labels, stage hot sector labels, leading stock labels, and important company event labels reaches a preset threshold, the topical label of this type is matched with the topical label vocabulary of the corresponding category that is pre-maintained and constructed, and the topical matching score of the topical label of this type and the matching item is calculated; wherein, if there are multiple matching items, the highest score is obtained as the topical matching score of the topical label of this type; wherein, the topical label vocabulary includes the macro policy label vocabulary, the real-time hot sector label vocabulary, the stage hot sector label vocabulary, the leading stock label vocabulary, and the important company event label vocabulary; The initial topic score of the information is obtained by weighted calculation of the topic matching scores of macro policy tags, real-time hot sector tags, stage hot sector tags, leading stock tags, important company event tags, information hot sector tag scores, tag number scores, and common tag scores, which is expressed as Initial topic score = macro policy label matching score × a1 + real-time hot sector label matching score × a2 + stage hot sector label matching score × a3 + leading stock label matching score × a4 + important company event label matching score × a5 + information hot sector label score × a6 + label number score × a7 + general label score × a8; in, is the topic weighting coefficient, and its value is determined by the time period of information release; the score of the hot information section label is determined by the frequency of occurrence of the hot information section label in the information; the number of labels score is determined by the number of content labels contained in the information; the common label score is determined by the weight of the content labels contained in the information; The topic score of the information is calculated based on the initial topic score and the fixed bonus of the information, and is expressed as Topic score = 100 / (1+EXP(2-0.017×initial topic score))+fixed bonus points; Among them, EXP represents the natural exponential function; the fixed bonus is used as a complementary strategy for topic quality evaluation, and its value is determined by the pre-written topic condition rule. When the information content hits the topic condition rule, a fixed bonus value is assigned; otherwise, no fixed bonus value is assigned.

3. The method according to claim 2, characterized in that The quality assessment of attractiveness specifically includes: Retrieve the title keywords of the news, match the title keywords with the pre-maintained and constructed title eye-catching word library and clickbait word library, and sum the calculated title eye-catching word matching score and clickbait matching score to get the news attractiveness score, which is expressed as Attractiveness score = title eye-catching words matching score + clickbait matching score.

4. The method according to claim 3, characterized in that The quality assessment of richness specifically includes: Based on the surface text structure characteristics of the information, the quality of the information title word count, paragraph number, paragraph length, sentence length, number of numbers, paragraph boldness, number of pictures and number of words is evaluated. The information richness score is calculated by comprehensively calculating the information title word count score, paragraph number score, paragraph length score, sentence length score, number of numbers score, paragraph boldness score, number of pictures score and number of words score, which is expressed as Richness score = ((title word count score + paragraph number score + paragraph length score + sentence length score + number ratio score + paragraph bold score + picture number score) / 7×0.4+0.6×text word count score).

5. The method according to claim 4, characterized in that The quality assessment of opinion, credibility, emotion and fluency includes: Retrieve the opinion expression words mentioned in the title and text of the news, match the opinion expression words with the pre-maintained opinion keyword word library and opinion regular table, and sum the calculated opinion keyword matching score and opinion regular matching score to obtain the opinion score of the news, which is expressed as Opinion score = opinion keyword matching score + opinion regular matching score; Retrieve the credibility expression words mentioned in the text of the news and obtain the media source of the news, match the credibility expression words with the credibility keyword word library constructed in advance, and calculate the credibility keyword matching score, match the media source of the news with the media source library constructed in advance, and calculate the media source matching score, perform weighted summation of the credibility keyword matching score and the media source matching score, and obtain the credibility score of the news, which is expressed as Credibility score = media source matching score × 0.7 + credibility keyword matching score × 0.3; Retrieve the emotional expressions mentioned in the text and title of the news, match the emotional expressions in the text and title with the emotional word database maintained and constructed in advance, and perform weighted summation of the calculated text emotional word matching score and title emotional word matching score to obtain the emotional score of the news, which is expressed as Sentiment score = text sentiment word matching score × 0.7 + title sentiment word matching score × 0.3; The information is perplexity calculated according to a pre-constructed content fluency model to obtain the information fluency score; wherein the content fluency model is trained using a NN language model.

6. The method according to claim 5, characterized in that The timeliness evaluation specifically includes: Extract the content tags of the information and assign corresponding weights to each type of content tag according to its importance. When the weight of a type of content tag reaches a certain threshold, match the content tag with the pre-maintained and constructed tag timeliness vocabulary, and calculate the timeliness matching score between the content tag and the matching item. If there are multiple matching items, the highest score is taken as the timeliness matching score of the content tag. Retrieve the timeliness keywords mentioned in the title and text of the news, match the timeliness keywords with the pre-maintained and constructed timeliness keyword thesaurus, and calculate the title timeliness matching score of the title timeliness keywords and the matching items, and the text timeliness matching score of the text timeliness keywords and the matching items; if there are multiple matching items, add up the multiple title or text timeliness matching scores to obtain the final title timeliness matching score and text timeliness matching score; The timeliness evaluation score of the information is calculated by combining the timeliness matching score of the content tag, the timeliness matching score of the title, and the timeliness matching score of the text, which is expressed as Timeliness evaluation score = 100 / (1+EXP(2.2-0.025*(timeliness matching score of content tag + timeliness matching score of title + timeliness matching score of text))); Wherein, EXP represents the natural exponential function.

7. The method according to claim 1, characterized in that The multi-dimensional deep feature quality evaluation score of the information is downgraded according to the category to which the information belongs, and the downgraded multi-dimensional deep feature quality evaluation score is obtained, which is expressed as The multi-dimensional deep feature quality evaluation score after downgrading = information downgrading coefficient × (topic score × topic coefficient + attractiveness score × attractiveness coefficient + credibility score × credibility coefficient + opinion score × opinion coefficient + fluency score × fluency coefficient + emotion score × emotion coefficient + richness score × richness coefficient); The value of the information weight reduction coefficient is determined by the category to which the information belongs and the manually maintained weight reduction regular rules. When the category to which the information belongs matches any of non-financial information, advertising soft article information, standardized information and announcement information, or when the title or text content of the information matches any of the weight reduction regular rules, the information weight reduction coefficient is assigned a value; if multiple assignment conditions are met at the same time, the minimum value is taken as the final value of the information weight reduction coefficient; Among them, the values ​​of the topic coefficient, attractiveness coefficient, richness coefficient, opinion coefficient, credibility coefficient, emotion coefficient and fluency coefficient are determined by the number of words in the information body and the time period of information release.

8. A device for evaluating the quality of information content in the field of finance and economics, characterized in that: The device comprises: The content label extraction and information classification module is used to extract content labels of multimedia source information in the financial and economic fields by using natural language processing technology, and obtain the category of each piece of information by inputting the multimedia source information into a pre-built financial / non-financial information classification model, an advertising soft text information classification model, and a standardized information classification model; wherein the financial / non-financial information classification model labels the multimedia source information as financial information and non-financial information; the advertising soft text information classification model labels the multimedia source information as advertising soft text information and non-advertising soft text information; the standardized information classification model labels the multimedia source information as standardized information, announcement information, and general information; The quality assessment module is used to perform multi-dimensional deep feature quality assessment and timeliness evaluation on each piece of information based on the content label and surface text structure characteristics of the information, and to downgrade the multi-dimensional deep feature quality assessment score of the information according to the category to which the information belongs, and to achieve a comprehensive quality assessment of the information content based on the downgraded multi-dimensional deep feature quality assessment score and timeliness evaluation score; wherein the multi-dimensional deep feature quality assessment includes quality assessment of topicality, attractiveness, richness, opinion, credibility, emotion and fluency.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images