Text processing method and device, electronic device and computer-readable storage medium

Abstract

The present application provides a text processing method and device, an electronic device, and a computer-readable storage medium. The method includes: acquiring text data; generating a first text vector and a second text vector respectively containing different feature components, performing feature joint processing to generate a feature joint vector; calculating a first feature vector for the first text vector; calculating a second feature vector according to the second text vector, the first feature vector, and the feature joint vector; determining the type of content to which the text data corresponding to the first text vector and the second text vector is directed according to the second feature vector. The embodiment of the present application improves the accuracy of rumor recognition by additionally setting a fusion layer of the user opinion model and the rumor recognition model, and inputting the input result of the fusion layer together with the output result of the user opinion model into the rumor recognition model as a feature reference for the rumor recognition model to recognize the corresponding text.

Description

Technical Field

[0001] The present application relates to the field of text processing, and in particular to a text processing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of Internet technology, people are increasingly fond of reading various news on the Internet. Therefore, with the increase in the demand for online reading, various information release channels have emerged on the Internet for publishing news, and many users will also express their opinions after reading the news, such as forwarding, commenting, liking, etc., so that the news spreads quickly while also spreading the views of the reading users, which greatly facilitates users to obtain information. However, due to irresponsible forwarding or malicious comments, false news, rumors and other erroneous views will also be widely spread, causing great harm to society. Summary of the invention

[0003] The embodiments of the present application provide a text processing method and device, an electronic device, and a computer-readable storage medium to solve the defect of inaccurate rumor type judgment in the prior art.

[0004] To achieve the above object, an embodiment of the present application provides a text processing method, characterized in that the text processing method includes:

[0005] Get text data;

[0006] Based on the text data, respectively generate a first text vector and a second text vector containing different feature components;

[0007] Performing feature joint processing on the first text vector and the second text vector to generate a feature joint vector;

[0008] Calculating a first feature vector for the first text vector, the first feature vector representing the user's position on the text data corresponding to the first text vector;

[0009] Calculate a second feature vector according to the second text vector, the first feature vector and the feature joint vector;

[0010] The type of content to which the text data corresponding to the first text vector and the second text vector refers is determined according to the second feature vector.

[0011] The present application also provides a text processing device, including:

[0012] An acquisition module, used to acquire text data;

[0013] A first vector generating module, used for generating a first text vector and a second text vector respectively containing different characteristic components based on the text data;

[0014] A joint processing module, used for performing feature joint processing on the first text vector and the second text vector to generate a feature joint vector;

[0015] a feature vector calculation module, configured to calculate a first feature vector for the first text vector, the first feature vector representing the user's position on the text data corresponding to the first text vector, and to calculate a second feature vector based on the second text vector, the first feature vector and the feature joint vector;

[0016] A determination module is used to determine the type of content to which the text data corresponding to the first text vector and the second text vector refers based on the second feature vector.

[0017] The present application also provides an electronic device, including:

[0018] Memory, used to store programs;

[0019] The processor is used to run the program stored in the memory to execute the text processing method provided in the embodiment of the present application.

[0020] An embodiment of the present application also provides a computer-readable storage medium on which a computer program executable by a processor is stored, wherein the program, when executed by the processor, implements the text processing method provided in the embodiment of the present application.

[0021] The text processing method and device, electronic device, and computer-readable storage medium provided in the embodiments of the present application can improve the influence of the output result of the user opinion model on the rumor identification result and improve the accuracy of rumor identification by additionally setting a fusion layer of the user opinion model and the rumor identification model, and inputting the input result of the fusion layer together with the output result of the user opinion model as a feature reference for the rumor identification model to identify the corresponding text into the rumor identification model. In particular, during model training, the training expectations of the rumor identification model can also be input into the fusion layer for training, so that compared with the method of simply training user opinion recognition, the user opinion recognition algorithm can be more suitable for identifying rumors, and the training results of the two models of user opinion recognition and rumor recognition can be unified, so that in actual use, the output result of the fusion layer can be more conducive to the accuracy of the recognition result of the corresponding text by the rumor identification layer.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0024] Figure 1 A schematic diagram of a scenario of a text processing system according to an embodiment of the present application;

[0025] Figure 2 A flowchart of a text processing method according to an embodiment of the present application;

[0026] Figure 3 is a flowchart of a text processing method according to another embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a text processing device according to one embodiment of the present application; and

[0028] Figure 5 It is a schematic structural diagram of an electronic device according to one embodiment of the present application. DETAILED DESCRIPTION

[0029] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] In the prior art, the rumor database is usually constructed using the identified historical rumors, so that when a new message is released, the new message can be compared with the historical rumors in the historical database, for example, the similarity between the new message and the historical rumor is calculated, so that the message with a similarity greater than a certain threshold is identified as a rumor. However, this method can only identify rumors that are the same or similar to the historical rumors in the rumor database, and new rumors involving new topics or scenes cannot be identified due to the absence of similar historical data. In particular, the speed at which information is generated and disseminated on the Internet is currently very alarming, so the accumulation of historical data can no longer keep up with the speed at which new messages are generated, which greatly reduces the practicality of this solution of comparing with historical rumors.

[0031] In this regard, a rumor identification scheme has emerged in the prior art. In addition to analyzing the input message using a neural network learning model trained with historical rumor data, for example, in addition to the above-mentioned calculation of the similarity with the historical rumor, the scheme further identifies the user's point of view in the message, thereby using the point of view identification result as one of the features of the rumor identification model as the input of the rumor identification model to assist the rumor identification model in identifying new messages. However, in this process, the identification of the user's point of view is carried out through a completely independent point of view identification model, and the rumor identification model only uses the point of view identification result output by the point of view identification model. In other words, the two models are completely independent of each other, and their training is also completely separate and independent. Therefore, the scheme cannot uniformly train the two models, so that the identification training of the user's point of view does not use the training content of the rumor identification model, and thus the improvement of the recognition accuracy of the rumor identification model by the user's point of view is very limited.

[0032] Figure 1 Schematic diagram of a scenario of a text processing system according to an embodiment of the present application. In actual applications, the texts published by users may be about various topics, some of which may be related to the users themselves, or to their lives, or may be social topics. Therefore, in this case, multiple users may publish multiple feedbacks on the same topic, such as the same content. Therefore, on the Internet, if a large number of texts published on a specific topic suddenly appear, then this topic may be a hot topic, and if the topic is a rumor, it may cause a very bad social impact. Therefore, when a large number of feedbacks published by multiple users on a certain topic are monitored on the Internet, rumor identification processing for the topic or the original published text can be triggered. For example, Figure 1As shown in , event 1 occurs in real life. In response, user 1 may post feedback 1 on the Internet for his own benefit or out of boredom or other reasons. The feedback 1 distorts and fabricates the truth of event 1. In this case, since there are other users, such as Figure 1 User 2 and User 3 in the event 1 are also concerned about event 1. Therefore, when reading feedback 1 posted by user 1, they are attracted by the fabricated content therein, and forward and / or add their own comments unintentionally or intentionally, thereby forming feedback 2 and feedback 3. Therefore, feedback 1, feedback 2 and feedback 3 form a message propagation link for event 1. The formation of such a message propagation link can be regarded as a possible cause of rumors in daily life, because the spreadability is one of the characteristics of rumors. Therefore, in this case, the rumor identification processing of the message on such a message propagation link can be triggered. In the prior art, as mentioned above, the above-mentioned feedback 1-3 is usually input into a rumor database composed of various texts that have been confirmed as rumors for identification. Such identification can be a direct text comparison, but it is more based on the calculation of a machine learning network. For example, an LSTM neural network is used to calculate the similarity between the input feedback 1-3 and the rumor text in the rumor database. For example, the input feedback 1-3 is converted into text vector 1-3 by word embedding processing (WE), and the similarity is calculated for each rumor text vector in the rumor database, and the similarity exceeding the predetermined threshold is sorted to determine which of the historical rumors the input feedback 1-3 is similar to. As described above, such processing is relatively effective for identifying rumors that have appeared in history or similar rumors. However, for relatively new rumor texts, such as the type of new events that have only appeared in recent years, they cannot be matched due to the low similarity with historical rumors. In this regard, it has also appeared in the prior art that, on the basis of calculating the similarity with historical rumors, when using LSTM neural networks for time-series-based machine learning calculations, the user's views on each input text calculated by the user's viewpoint machine learning model can be additionally considered, such as approval, indicating that it is true; opposition, indicating that it is considered to be a rumor, etc. By additionally considering the user's attitude towards the text or event involved expressed by the text when comparing each input text with historical rumors, the problem of being unable to identify new types of rumors by relying solely on historical rumor samples can be solved to a certain extent. However, in this process, the determination of the attitude expressed by the user in the text is based on a user attitude model that is completely independent of the rumor recognition model. Therefore, although the results of the user opinion model are also used when identifying rumors, the training corpus of the rumor machine learning model cannot be used to uniformly train the user opinion model. Therefore, it is impossible to improve the impact of its output results on the recognition accuracy of the rumor machine learning model.

[0033] To this end, in an embodiment of the present application, a fusion layer is added, that is, in addition to the two independent models of the user opinion model and the rumor recognition model, a fusion layer of the two models is also set up. In this layer, the training corpus of the user opinion and rumor recognition can be input together for training, and when calculating the possibility or similarity of each input text being a rumor, in addition to the output result of the user opinion model, the input result of the fusion layer can also be additionally input into the rumor recognition model as a feature reference to calculate the possibility that the text is a rumor.

[0034] Therefore, through the scheme of the embodiment of the present application, it is possible to set up an additional fusion layer of the user opinion model and the rumor recognition model, and input the input result of the fusion layer together with the output result of the user opinion model as a feature reference for the rumor recognition model to recognize the corresponding text, thereby improving the influence of the output result of the user opinion model on the rumor recognition result and improving the accuracy of rumor recognition. In particular, during model training, the training expectations of the rumor recognition model can also be input into the fusion layer for training, thereby making the user opinion recognition algorithm more suitable for identifying rumors compared to simply training user opinion recognition, and obtaining the training results that unify the two models of user opinion recognition and rumor recognition, so that in actual use, the output result of the fusion layer can be more conducive to the accuracy of the recognition result of the corresponding text by the rumor recognition layer.

[0035] Next reference Figures 2 to 5 A text processing method, a text processing device, an electronic device, and a computer-readable storage medium according to embodiments of the present application are described in detail.

[0036] Figure 2 FIG. 1 is a flow chart of a text processing method according to an embodiment of the present application. Figure 2 As shown in, according to an embodiment of the present application, the text processing method of the present application may include:

[0037] S201, obtaining text data.

[0038] In the embodiment of the present application, various text data related to the event can be obtained from various data sources, for example, various text data related to the event can be obtained from a database of an Internet forum, a database of a social software, or uploaded by a user. In the embodiment of the present application, the obtained text data can be ensured to be text data related to the event by selecting a database as a source for obtaining text data, or the text data can be selected after the text data is obtained.

[0039] In addition, in real life, usually with the occurrence of an event, many texts will appear on the Internet for a certain event or a certain content or a certain theme, that is, different users will express their views on hot issues or events or themes of interest. Therefore, usually, a series of texts for a certain event or a certain content, i.e., a text data chain, will be formed. Therefore, in this case, in step S201, such multiple text data can also be obtained to form a text data chain for the same content or theme, and each text data can be located at each node on the text data chain.

[0040] S202: Generate a first text vector and a second text vector containing different feature components based on the text data.

[0041] After the text data is obtained, in step S202, the word vectors contained in the text data can be used for each text data using processing such as word embedding. In an embodiment of the present application, when identifying the type of content targeted by the text data, such as whether it is a rumor, the user's opinion on the content represented in the text data can be referred to. For example, in an embodiment of the present application, when the user publishes the text data, he usually expresses his opinion on the targeted content through the text. For example, in step S201, the text published by the user for an event or content can be collected. The text can semantically express the user's opinion on the event or content, such as support, that is, agreeing with the content and believing it to be true; veto, that is, believing that the content is false news; inquiry, that is, expressing the desire to further understand the situation; and comment, that is, only commenting on the content through the text data, but the text data does not express the tendency of whether the content is true. Therefore, in the present application, a first text vector for judging the user's opinion can be generated for the text data obtained in step S201, which may include various feature components that are helpful in judging the user's opinion in addition to the word vector of the text data itself. For example, the feature components of the content targeted by the text data may be included in the first text vector, such as the text vector or content vector of another text data cited or targeted by the user when publishing the text data. In addition, a vector of the attributes of the text data itself may be included in the first text vector, such as a vector generated by extracting different attributes from the text data, for example, whether the text data contains a person's name, whether it contains pictures, whether it contains links, etc., so that this information can be included when generating the first text vector to facilitate the subsequent judgment of the user's opinion.

[0042] In addition, in the present application, a second text vector for the type of content targeted by the text data can be generated for the text data obtained in step S201, wherein, similarly, in addition to the word vector of the text data itself, various feature components that are helpful in judging the type of content can also be included. For example, the user feature vector of the user who published the text data can be included in the second text vector. That is, various attribute information is extracted from the portrait information of the user who uploaded or published the text data to obtain the portrait vector of the user. For example, the attribute information in the user's portrait information may include the length of time the user account has been registered, whether it is a certified account, the number of text data published by the user, the number of other users who follow the user (i.e., the number of fans), the industry information of the user, and the like. The credibility of the user can be reflected through these attribute information, so that it can be used as a reference when judging the text data published by the user.

[0043] S203: Perform feature joint processing on the first text vector and the second text vector to generate a feature joint vector.

[0044] In an embodiment of the present application, feature joint processing may be performed on the first text vector used to calculate the user's opinion and the second text vector used to determine the type of content to generate a feature joint vector containing various feature information used to calculate the user's opinion and various feature information used to determine the type of content. For example, when generating the feature vector in step S203, various vector generation methods in the art may be used, such as LSTM models, etc. In the process, the attention mechanism may be further used to calculate the importance, i.e., weight, of each word vector or feature information of the text data in the process of generating the text vector, so that the generated feature vector contains more relatively important information.

[0045] In addition, in the present application, generating a feature joint vector based on the first text vector and the second text vector can be regarded as sharing various feature information for calculating user opinions and judging content types, thereby not only being able to generate feature vectors containing feature information for different purposes, but also being able to train the LTSM model by using sample data, thereby optimizing the parameters for generating feature vectors.

[0046] S204: Calculate a first feature vector for the first text vector.

[0047] In the present application, the user's opinion can be calculated based on the first text vector generated in step S202, so as to obtain a feature vector containing the user's opinion type. In an embodiment of the present application, LSTM can be used to generate a feature vector, so as to obtain a feature vector representing the user's opinion (position) type (for example, support, veto, inquiry or comment) and a corresponding implicit vector, and when multiple text data are obtained in step S201, the implicit vector can be used to pass the calculation result of the current vector to the calculation of the feature vector of the text data of the next node on the text data chain.

[0048] S205, calculating a second feature vector according to the second text vector, the first feature vector and the feature joint vector.

[0049] In the embodiment of the present application, after the feature joint vector is obtained in step S203 and the feature vector representing the user's opinion is obtained in step S204, these two vectors can be processed together with the second text vector obtained based on the text data in step S202 for content type identification. For example, compared with the prior art, in the embodiment of the present application, when judging the type of content targeted by the text data, the opinions expressed by the user in the text data are considered, that is, the opinions (positions) for the content targeted by the text data, and a feature joint vector that integrates the feature components for calculating the user's opinion and the feature components for judging the type of content is additionally used as input, so that the comprehensive vector that shares the features for different calculation purposes (user's opinion and judging whether the content is a rumor) can be fully utilized.

[0050] S206: Determine, according to the second feature vector, the type of content to which the text data corresponding to the first text vector and the second text vector refers.

[0051] In the embodiment of the present application, after the second feature vector is obtained in step S205, such feature vector can be calculated in step S206, and the obtained result identifies the type of content targeted by the text data. For example, the type can be: true content, false content, cannot be verified yet, etc.). In the embodiment of the present application, when multiple text data forming a text data chain are obtained in step S201, the softmax function can be used in step S206 to calculate the second feature vectors of the multiple text data on the data chain to determine the type of content targeted by these text data.

[0052] Therefore, through the scheme of the embodiment of the present application, it is possible to set up an additional fusion layer of the user opinion model and the rumor recognition model, and input the input result of the fusion layer together with the output result of the user opinion model as a feature reference for the rumor recognition model to recognize the corresponding text, thereby improving the influence of the output result of the user opinion model on the rumor recognition result and improving the accuracy of rumor recognition. In particular, during model training, the training expectations of the rumor recognition model can also be input into the fusion layer for training, thereby making the user opinion recognition algorithm more suitable for identifying rumors compared to simply training user opinion recognition, and obtaining the training results that unify the two models of user opinion recognition and rumor recognition, so that in actual use, the output result of the fusion layer can be more conducive to the accuracy of the recognition result of the corresponding text by the rumor recognition layer.

[0053] Figure 3 It is a flowchart of a text processing method according to another embodiment of the present application.

[0054] like Figure 3 As shown in, according to an embodiment of the present application, the text processing method of the present application may include:

[0055] S301, obtaining text data.

[0056] In the embodiment of the present application, various text data related to the event can be obtained from various data sources, for example, various text data related to the event can be obtained from a database of an Internet forum, a database of a social software, or uploaded by a user. In the embodiment of the present application, the obtained text data can be ensured to be text data related to the event by selecting a database as a source for obtaining text data, or the text data can be selected after the text data is obtained.

[0057] In addition, in real life, usually with the occurrence of an event, a lot of texts will appear on the Internet for a certain event or a certain content or a certain theme, that is, different users will express their views on hot issues or events or themes of interest. Therefore, usually, a series of feedbacks for a certain event or a certain content will be formed, so that a text data chain can be formed. Therefore, in this case, multiple feedbacks for the same content can be obtained in step S301, and these multiple feedbacks can jointly constitute a text data chain for the same content and each feedback is respectively located on each node on the text data chain to form a text data chain for the same content or theme.

[0058] S302 , generating a first text vector according to the text vectors of the characters in the text data, the attribute vectors of the attributes of the text data, and the content vectors of the contents to which the text data corresponds.

[0059] After obtaining the feedback, in step S302, a first text vector can be generated based on the obtained multiple feedbacks (for example, its word vector and other relevant feature information). For example, in an embodiment of the present application, when a user publishes text data, he usually expresses his views on the content through the text, that is, forming feedback on the content. For example, in step S301, the text published by the user for an event or content, that is, feedback on the content, can be collected. The feedback can semantically express the user's views on the event or content, such as support, that is, agreeing with the content and believing it to be true; veto, that is, believing that the content is false news; inquiry, that is, expressing the desire to further understand the situation; and comment, that is, through the feedback, only the content is commented on, but the feedback does not express the tendency of whether the content is true.

[0060] Therefore, in step S302, a first text vector for judging the user's opinion can be generated for the feedback obtained in step S301, which may include various feature components that are helpful in judging the user's opinion in addition to the word vectors of the multiple feedbacks themselves. For example, the first text vector may include feature components of the content targeted by the multiple feedbacks, such as the text vector or content vector of another text data cited or targeted by the user when publishing the text data. In addition, the first text vector may include a vector of the attributes of the text data itself, such as a vector generated by extracting different attributes from the text data, for example, whether the text data contains a person's name, whether it contains pictures, whether it contains links, etc., so that this information can be included when generating the first text vector to facilitate the subsequent judgment of the user's opinion.

[0061] S303 , generating a second text vector according to the text vectors of the characters in the text data and the user attribute vectors of the user attributes of the user who publishes the text data.

[0062] In the present application, for the text data obtained in step S301, for example, multiple feedbacks can be used to generate a second text vector for the type of content targeted by the text data, wherein, in addition to the word vector of the text data itself, various feature components that are helpful in judging the type of content can also be included. For example, the user feature vector of the user who published the text data can be included in the second text vector. That is, various attribute information is extracted from the portrait information of the user who uploaded or published the text data to obtain the portrait vector of the user. For example, the attribute information in the user's portrait information may include the length of time the user account has been registered, whether it is a certified account, the number of text data published by the user, the number of other users who follow the user (i.e., the number of fans), the industry information of the user, and the like. The credibility of the user can be reflected through these attribute information, so that it can be used as a reference when judging the text data published by the user.

[0063] Of course, when the text data is a plurality of feedbacks on the same content, the user feature vector of the user who provided each feedback may be included when generating the second text vector based on each feedback, thereby facilitating the determination of the attributes of each user who provided each feedback.

[0064] S304: Perform feature joint processing on the first text vector and the second text vector to generate a feature joint vector.

[0065] In an embodiment of the present application, feature joint processing may be performed on the first text vector used to calculate the user's opinion and the second text vector used to determine the type of content to generate a feature joint vector containing various feature information used to calculate the user's opinion and various feature information used to determine the type of content. For example, when generating the feature vector in step S203, various vector generation methods in the art may be used, such as LSTM models, etc. In the process, the attention mechanism may be further used to calculate the importance, i.e., weight, of each word vector or feature information of the text data in the process of generating the text vector, so that the generated feature vector contains more relatively important information.

[0066] In addition, in the present application, generating a feature joint vector based on the first text vector and the second text vector can be regarded as sharing various feature information for calculating user opinions and judging content types, thereby not only being able to generate feature vectors containing feature information for different purposes, but also being able to train the LTSM model by using sample data, thereby optimizing the parameters for generating feature vectors.

[0067] S305: Calculate a first feature vector according to the feature joint vector and the first text vector.

[0068] In the present application, the user's opinion can be calculated based on the first text vector generated in step S302 and the feature joint vector generated in step S304, so as to obtain a feature vector containing the user's opinion type. In the embodiment of the present application, since a feature vector that integrates the feature information for the user's opinion and the feature information for determining the content type is additionally considered when calculating the user's opinion. In particular, since such a comprehensive shared model can be trained using samples for different purposes, a feature joint vector containing more useful information can be obtained when used, so that the first feature vector can carry more information when subsequently used to determine the user's opinion.

[0069] S306, calculating a second feature vector according to the second text vector, the first feature vector and the feature joint vector.

[0070] In the embodiment of the present application, after the feature joint vector is obtained in step S305 and the feature vector representing the user's point of view is obtained in step S304, these two vectors can be processed together with the second text vector obtained based on the text data in step S302 for content type recognition. For example, compared with the prior art, in the embodiment of the present application, when judging the type of content targeted by the text data, the opinions expressed by the user in the text data are considered, that is, the opinions (positions) for the content targeted by the text data, and a feature joint vector that integrates the feature components for calculating the user's point of view and the feature components for judging the type of content is additionally used as input, so that the comprehensive vector that shares the features for different calculation purposes (user's point of view and judging whether the content is a rumor) can be fully utilized.

[0071] S307: Calculate a weight value of each feedback according to the second text vectors of the multiple feedbacks.

[0072] In an embodiment of the present application, the attention mechanism can be calculated for the second text vectors of multiple feedbacks on the same content obtained in the text data chain in step S303 to obtain each feedback, that is, the weight value of each second text vector relative to other second text vectors. The weight value can represent each node on the text data chain, that is, each feedback has a greater effect on the final content type judgment, so that the feedback can have a larger weight value.

[0073] S308 , performing abstract calculation according to the second feature vector of each feedback in the text data to obtain an abstract feature vector identifying the type of content targeted by the feedback.

[0074] In addition, when the weight value of each second text vector is calculated in step S307, the text processing method of the present application may include step S3081, in which an abstract calculation is performed based on the second feature vector and weight value of each feedback in the text data to obtain an abstract feature vector representing the type of content to which the feedback is directed.

[0075] S309: Calculate the type of content targeted by the text data according to the abstract feature vector.

[0076] In an embodiment of the present application, after the second feature vector is obtained in step S306, such feature vector can be abstractly calculated in step S307 to obtain an abstract feature vector. The abstract feature vector can identify the type of content targeted by the text data. For example, the type can be: true content, false content, cannot be verified yet, etc.). In an embodiment of the present application, step S307 can use a softmax function to calculate the abstract feature vector of the second feature vector of multiple text data on the data chain to determine the type of content targeted by these text data.

[0077] Therefore, through the scheme of the embodiment of the present application, it is possible to set up an additional fusion layer of the user opinion model and the rumor recognition model, and input the input result of the fusion layer together with the output result of the user opinion model as a feature reference for the rumor recognition model to recognize the corresponding text, thereby improving the influence of the output result of the user opinion model on the rumor recognition result and improving the accuracy of rumor recognition. In particular, during model training, the training expectations of the rumor recognition model can also be input into the fusion layer for training, thereby making the user opinion recognition algorithm more suitable for identifying rumors compared to simply training user opinion recognition, and obtaining the training results that unify the two models of user opinion recognition and rumor recognition, so that in actual use, the output result of the fusion layer can be more conducive to the accuracy of the recognition result of the corresponding text by the rumor recognition layer.

[0078] Figure 4 FIG. 1 is a schematic diagram of a text processing device according to an embodiment of the present application. Figure 2 and Figure 3 The method steps shown. Figure 4 As shown, the text processing device of the embodiment of the present application includes: an acquisition module 401, a first vector generation module 402, a joint processing module 403, a feature vector calculation module 404 and a determination module 405.

[0079] The acquisition module 401 may be used to acquire text data.

[0080] In the embodiment of the present application, the acquisition module 401 can obtain various text data related to the event from various data sources, for example, various text data related to the event can be obtained from a database of an Internet forum, a database of a social software, or uploaded by a user. In the embodiment of the present application, the acquisition module 401 can also ensure that the acquired text data is text data related to the event by selecting a database as a source for acquiring text data, or the text data can be selected after the text data is acquired.

[0081] In addition, in real life, usually accompanied by the occurrence of a certain event, a lot of texts, i.e. feedback, will appear on the Internet for a certain event or a certain content or a certain theme, that is, different users will express their opinions for hot issues or events or themes of interest, thereby forming feedback for the content of the event or theme. Therefore, usually, a series of feedbacks for a certain event or a certain content will be formed, thereby a text data chain can be formed. Therefore, in this case, acquisition module 401 can obtain such multiple feedbacks, to form a text data chain for the same content or theme, and each feedback can be on each node on the text data chain.

[0082] The first vector generating module 402 may be used to generate a first text vector and a second text vector containing different feature components based on the text data.

[0083] After the acquisition module 401 acquires the text data, the first vector generation module 402 can use a process such as word embedding for each text data to embed the word vectors contained in the text data. In an embodiment of the present application, when identifying the type of content targeted by the text data, such as whether it is a rumor, the user's opinion on the content represented in the text data can be referred to. For example, in an embodiment of the present application, when the user publishes the text data, he usually expresses his opinion on the targeted content through the text. For example, the acquisition module 401 can collect the text published by the user for an event or content, and the text can semantically express the user's opinion on the event or content, such as support, that is, agreeing with the content and believing it to be true; veto, that is, believing that the content is false news; inquiry, that is, expressing the desire to further understand the situation; and comment, that is, only commenting on the content through the text data, but the text data does not express the tendency of whether the content is true. Therefore, in the present application, the first vector generation module 402 may include a first text vector generation unit 4021, which may generate a first text vector for judging the user's point of view for the text data obtained by the acquisition module 401, wherein in addition to the word vector of the text data itself, various feature components that are helpful in judging the user's point of view may be included. For example, the feature components of the content targeted by the text data may be included in the first text vector, such as the text vector or content vector of another text data cited or targeted by the user when publishing the text data. In addition, the vector of the attribute of the text data itself may be included in the first text vector, such as a vector generated by extracting different attributes from the text data, for example, whether the text data contains a person's name, whether it contains a picture, whether it contains a link, etc., so that the first text vector generation unit 4021 of the first vector generation module 402 may include this information when generating the first text vector, so as to facilitate the subsequent judgment of the user's point of view.

[0084] The first vector generation module 402 may also include a second text vector generation unit 4022, which may generate a second text vector for the type of content targeted by the text data for the text data acquired by the acquisition module 401, for example, for multiple feedbacks of the same content, wherein, in addition to the word vector of the text data itself, the second text vector generation unit 4022 of the first vector generation module 402 may also include various feature components that are helpful for determining the type of content in the second text vector. For example, the user feature vector of the user who published the text data may be included in the second text vector. That is, various attribute information is extracted from the portrait information of the user who uploaded or published the text data to obtain the portrait vector of the user. For example, the attribute information in the user's portrait information may include the length of time of user account registration, whether it is a certified account, the number of text data published by the user, the number of other users who follow the user (i.e., the number of fans), the industry information of the user, etc. The credibility of the user can be reflected through these attribute information, so that it can be used as a reference when judging the text data published by the user.

[0085] Of course, when the text data is a plurality of feedbacks on the same content, the user feature vector of the user who provided each feedback may be included when generating the second text vector based on each feedback, thereby facilitating the determination of the attributes of each user who provided each feedback.

[0086] The joint processing module 403 may be configured to perform feature joint processing on the first text vector and the second text vector to generate a feature joint vector.

[0087] In the embodiment of the present application, the joint processing module 403 can perform feature joint processing on the first text vector used to calculate the user's opinion and the second text vector used to determine the type of content, so as to generate a feature joint vector containing various feature information used to calculate the user's opinion and various feature information used to determine the type of content. For example, when generating the feature vector, various vector generation methods in the art can be used.

[0088] In addition, in the present application, the joint processing module 403 generates a feature joint vector based on the first text vector and the second text vector, which can be regarded as a shared processing of various feature information for calculating user opinions and judging content types, thereby not only being able to generate feature vectors containing feature information for different purposes, but also being able to train the LTSM model by using sample data, thereby optimizing the parameters for generating feature vectors.

[0089] The feature vector calculation module 404 may be configured to calculate a first feature vector for the first text vector, and calculate a second feature vector for the first text vector according to the second text vector, the first feature vector and the feature joint vector.

[0090] For example, in the present application, the feature vector calculation module 404 may include a first feature vector calculation unit 4041 and a second feature vector calculation unit 4042 .

[0091] In the present application, the first feature vector calculation unit 4041 can calculate the user's opinion for the first text vector generated by the first vector generation module 402, so as to obtain a feature vector containing the user's opinion type. In an embodiment of the present application, LSTM can be used to generate feature vectors, so as to obtain feature vectors representing the user's opinion (position) type (for example, support, veto, inquiry or comment) and corresponding implicit vectors, and when the acquisition module 401 obtains multiple text data, the first feature vector calculation unit 4041 can use the implicit vector to pass the calculation result of the current vector to the next node on the text data chain for use in the calculation of the feature vector of the text data.

[0092] The second feature vector calculation unit 4042 may be configured to calculate a second feature vector according to the second text vector, the first feature vector and the feature joint vector.

[0093] In an embodiment of the present application, after the joint processing module 403 obtains the feature joint vector and the first vector generation module 402 obtains the feature vector representing the user's opinion, the second feature vector calculation unit 4042 can perform content type identification processing on these two vectors together with the second text vector obtained by the first vector generation module 402, for example, the second text vector generation unit based on the text data.

[0094] The determination module 405 may be configured to determine, based on the second feature vector, the type of content to which the text data corresponding to the first text vector and the second text vector refers.

[0095] In the embodiment of the present application, the determination module 405 can calculate the second feature vector obtained by the second feature vector calculation unit 4042, and the obtained result identifies the type of content targeted by the text data. For example, the type can be: true content, false content, cannot be verified yet, etc.).

[0096] In the embodiment of the present application, the text processing apparatus may further include an attention calculation module 406, which is used to calculate the weight value of each feedback according to the second text vectors of multiple feedbacks.

[0097] In the embodiment of the present application, the attention calculation module 406 can perform an attention mechanism calculation on the second text vectors of multiple feedbacks on the same content on the text data chain obtained by the acquisition module 401 to obtain each feedback, that is, the weight value of each second text vector relative to other second text vectors. The weight value can represent each node on the text data chain, that is, each feedback has a greater effect on the final content type judgment, so that the feedback can have a larger weight value.

[0098] In the embodiment of the present application, the determination module 405 may further include an abstract calculation unit 4051 and a first calculation unit 4052. The abstract calculation unit 4051 may be used to perform an abstract calculation according to the second feature vector of each feedback in the text data to obtain an abstract feature vector identifying the type of content targeted by the feedback.

[0099] In addition, the attention calculation module 406 can calculate the weight value of each second text vector of the multiple text data acquired by the acquisition module 401, and the abstract calculation unit 4051 can perform abstract calculation based on the second feature vector and weight value of each of the multiple text data to obtain an abstract feature vector representing the type of content targeted by the text data.

[0100] The first calculation unit 4052 may be configured to calculate the type of content targeted by the text data according to the abstract feature vector.

[0101] Therefore, through the scheme of the embodiment of the present application, it is possible to set up an additional fusion layer of the user opinion model and the rumor recognition model, and input the input result of the fusion layer together with the output result of the user opinion model as a feature reference for the rumor recognition model to recognize the corresponding text, thereby improving the influence of the output result of the user opinion model on the rumor recognition result and improving the accuracy of rumor recognition. In particular, during model training, the training expectations of the rumor recognition model can also be input into the fusion layer for training, thereby making the user opinion recognition algorithm more suitable for identifying rumors compared to simply training user opinion recognition, and obtaining the training results that unify the two models of user opinion recognition and rumor recognition, so that in actual use, the output result of the fusion layer can be more conducive to the accuracy of the recognition result of the corresponding text by the rumor recognition layer.

[0102] The internal functions and structure of the text processing device are described above. The device can be implemented as an electronic device. Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device includes a memory 51 and a processor 52 .

[0103] The memory 51 is used to store programs. In addition to the above programs, the memory 51 can also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0104] The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0105] The processor 52 is not limited to a central processing unit (CPU), but may also be a processing chip such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an embedded neural network processor (NPU) or an artificial intelligence (AI) chip. The processor 52 is coupled to the memory 51 to execute a program stored in the memory 51, and when the program is run, any text processing method provided in the embodiments of the present application is executed.

[0106] Further, if Figure 5 As shown, the electronic device may also include: a communication component 53, a power component 54, an audio component 55, a display 56 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0107] The communication component 53 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 53 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 53 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0108] The power supply component 54 provides power to various components of the electronic device. The power supply component 54 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0109] The audio component 55 is configured to output and / or input audio signals. For example, the audio component 75 includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 51 or sent via the communication component 53. In some embodiments, the audio component 55 also includes a speaker for outputting audio signals.

[0110] The display 56 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0111] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A text processing method, characterized in that: include: Get text data; Based on the text data, a first text vector and a second text vector each containing different characteristic components are generated, wherein the first text vector is generated according to the text vector of the text data, the attribute vector of the attribute of the text data, and the content vector of the content to which the text data refers; and the second text vector is generated according to the text vector of the text data and the user attribute vector of the user attribute of the user who publishes the text data; Performing feature joint processing on the first text vector and the second text vector to generate a feature joint vector; Calculating a first feature vector for the first text vector, the first feature vector representing the user's position on the text data corresponding to the first text vector, wherein the calculating the first feature vector for the first text vector comprises: calculating the first feature vector according to the feature joint vector and the first text vector; Calculate a second feature vector according to the second text vector, the first feature vector and the feature joint vector; The type of content to which the text data corresponding to the first text vector and the second text vector refers is determined according to the second feature vector.

2. The text processing method according to claim 1, characterized in that: The text data is a plurality of feedbacks for the same content, part or all of the plurality of feedbacks constitute a text data chain for the same content, and each of the plurality of feedbacks constituting the text data chain is located at each node on the text data chain; then The determining, according to the second feature vector, the type of content targeted by the text data corresponding to the first text vector and the second text vector comprises: Performing an abstract calculation based on the second feature vector of each feedback in the text data to obtain an abstract feature vector identifying the type of content targeted by the feedback; The type of content targeted by the feedback is calculated according to the abstract feature vector.

3. The text processing method according to claim 2, characterized in that: The text processing method further comprises: Calculating a weight value of each feedback according to the second text vectors of the plurality of feedbacks, and The performing abstract calculation according to the second feature vector of each feedback in the text data to obtain a feature vector identifying the type of content targeted by the feedback comprises: Abstract calculation is performed based on the second feature vector and weight value of each feedback in the text data to obtain an abstract feature vector representing the type of content targeted by the feedback.

4. A text processing device, comprising: An acquisition module, used to acquire text data; A first vector generation module is used to generate a first text vector and a second text vector containing different characteristic components based on the text data, wherein the first vector generation module includes: a first text vector generation unit, which is used to generate the first text vector according to the text vector of the text data, the attribute vector of the attribute of the text data, and the content vector of the content targeted by the text data; a second text vector generation unit, which is used to generate the second text vector according to the text vector of the text data and the user attribute vector of the user attribute of the user who publishes the text data; A joint processing module, used for performing feature joint processing on the first text vector and the second text vector to generate a feature joint vector; a feature vector calculation module, configured to calculate a first feature vector for the first text vector, the first feature vector representing the user's position on the text data corresponding to the first text vector, and to calculate a second feature vector based on the second text vector, the first feature vector and the feature joint vector, wherein the feature vector calculation module comprises: a first feature vector calculation unit, configured to calculate the first feature vector based on the feature joint vector and the first text vector; A determination module is used to determine the type of content to which the text data corresponding to the first text vector and the second text vector refers based on the second feature vector.

5. The text processing device according to claim 4, characterized in that: The text data is a plurality of feedbacks for the same content, part or all of the plurality of feedbacks constitute a text data chain for the same content, and each of the plurality of feedbacks constituting the text data chain is located at each node on the text data chain; but The determination module comprises: An abstract calculation unit, configured to perform abstract calculation according to the second feature vector of each feedback in the text data, so as to obtain an abstract feature vector identifying the type of content targeted by the feedback; The first calculation unit is configured to calculate the type of content targeted by the feedback according to the abstract feature vector.

6. The text processing device according to claim 4, characterized in that: The feature vector calculation module includes: The second feature vector calculation unit is used to calculate a second feature vector according to the second text vector, the first feature vector and the feature joint vector.

7. The text processing device according to claim 5, characterized in that: The text processing device further comprises: an attention calculation module, configured to calculate a weight value of each feedback according to the second text vectors of the plurality of feedbacks, and The abstract computing unit is further configured to: Abstract calculation is performed based on the second feature vector and weight value of each feedback in the text data to obtain an abstract feature vector representing the type of content targeted by the feedback.

8. An electronic device, wherein: include: Memory, used to store programs; A processor, configured to run the program stored in the memory to execute the text processing method as claimed in any one of claims 1 to 3.

9. A computer-readable storage medium having stored thereon a computer program executable by a processor, wherein: When the program is executed by a processor, the text processing method according to any one of claims 1 to 3 is implemented.

Images