A text data processing method and device

Abstract

The present application discloses a text data processing method and device thereof, the method comprising: converting the acquired text data into candidate phrases in a predetermined manner; determining whether a word segment in each candidate phrase belongs to at least one preset dictionary; if it belongs to at least one preset dictionary, determining the corresponding candidate phrase as a visual phrase. The present application makes the semantics expressed by the user more readable and semantically richer.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text data processing method and device thereof. Background Art

[0002] With the development of technology, data visualization can use graphical means to display relatively obscure data in a visual and interactive way, thereby vividly and intuitively expressing the information and laws contained in the data. Therefore, it is attracting more and more attention as a new research field.

[0003] As an important display method for data visualization, word cloud can visually highlight the "keywords" that appear frequently in online texts (for example, comments), forming a "keyword cloud layer" or "keyword rendering", thereby filtering out a large amount of text information. However, the word clouds generated by existing technologies are usually obscure and have a high degree of information overlap. Therefore, a technical solution that can generate highly readable and information-rich word clouds is needed.

[0004] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no statement is made, as to whether any of the above information may be applicable as prior art with respect to the present disclosure. Summary of the invention

[0005] The embodiments of the present application provide a text data processing method and a device thereof, aiming to at least solve the above-mentioned technical problems.

[0006] An embodiment of the present application also provides a text data processing method, which includes: converting the acquired text data into candidate phrases in a predetermined manner; determining whether the word segment in each candidate phrase belongs to at least one preset dictionary; if it belongs to at least one preset dictionary, determining the corresponding candidate phrase as a visual phrase.

[0007] An embodiment of the present application also provides a text data processing device, which includes: a processor; and a memory arranged to store computer executable instructions, and when the executable instructions are executed, the processor executes the above method.

[0008] The embodiment of the present application also provides a computer-readable storage medium on which computer instructions are stored, and the above method is implemented when the instructions are executed.

[0009] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0010] The text data processing method of the exemplary embodiment of the present application can extract visual phrases from text data using a preset dictionary, so that the presented word cloud has higher readability and richer semantics. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 is a diagram of a text data processing system according to an exemplary embodiment of the present application;

[0013] Figure 2 is a flowchart of steps of a text data processing method according to an exemplary embodiment of the present application;

[0014] Figure 3 is a flowchart of steps for generating at least one predetermined dictionary according to an exemplary embodiment of the present application;

[0015] Figure 4 is an example diagram of a word cloud provided according to an exemplary embodiment of the present application;

[0016] Figure 5 is a flowchart of steps of a text data processing method according to another exemplary embodiment of the present application;

[0017] Figure 6 is a block diagram of a text data processing apparatus according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0019] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0020] Figure 1 is a diagram of a text data processing system according to an exemplary embodiment of the present application. For descriptive purposes, the depicted architecture is only an example of a suitable environment and does not impose any limitations on the scope of use or functionality of the present application. The computing system should not be interpreted as Figure 1 There are no dependencies or requirements on any one or combination of components shown.

[0021] The principles of the present application can be operated using other general or special computing or communication environments or configurations. Examples of well-known computing systems, environments and configurations suitable for the present application include, but are not limited to, personal computers, servers, multiprocessor systems, microprocessor-based systems, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0022] In its most basic configuration, Figure 1 The text data processing system 100 in the embodiment includes at least: one or more clients 104 and a server 102 that performs text data processing on the text sent by the one or more clients 104. The server 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a storage device for storing data, and a transmission device for communicating with the client; the client 104 may include: a microprocessor MCU, a transmission device for communicating with the server, and a display device for interacting with the user. In the present application, a "system" may also be defined as any hardware component or combination of hardware components that can execute software, firmware, or microcode to implement functions. The text data processing system 100 may even be distributed to implement distributed functions.

[0023] As used herein, the term "module", "component" or "unit" may refer to a software object or routine executed on the configuration information updating system 100. The different components, modules, units, engines and services described herein may be implemented as objects or processes executed (e.g., as separate threads) on the text data processing method 100. Although the systems and methods described herein are preferably implemented in software, implementation in hardware or a combination of software and hardware is also possible and contemplated.

[0024] In actual processing, online commentators can create and publish comment content for a certain video through their respective clients 104, and the server 102 can, after obtaining the comment content published by each client, perform text data processing on these comment contents according to the exemplary embodiments of the present application, thereby generating a word cloud that can accurately express the opinions of the audience.

[0025] like Figure 1 As shown, the client 104 may not be directly connected to the server 102, that is, the client 104 may send text data corresponding to the comment content of the commentator on the played video to the corresponding video server 200, and then the video server 200 may send the text data to the server 102. Finally, the server 102 may perform text data processing on the text data according to the text data processing method of the exemplary embodiment of the present application to generate an appropriate word cloud for display.

[0026] The following will be combined Figures 2 to 3 The text data processing method executed by the server 102 is explained in detail.

[0027] Figure 2 1 is a flowchart of the steps of the text data processing method according to the exemplary embodiment of the present application. In the embodiment of the present invention, the execution subject of the text data processing method is a computing device with text data processing capability such as a terminal or a server, or multiple computing devices deployed in a distributed system.

[0028] Reference Figure 2 , the text data processing method includes: step S210, converting the acquired text data into candidate phrases in a predetermined manner. Specifically, first, after authorization, the text data in the web page can be obtained. Here, the text data refers to the text data in the Internet data resources, and the amount of these text data is huge and scattered in the pages of various websites or applications (APPs) on the Internet. That is to say, due to the difference in websites or applications, the sources of these text data are also different. For example, the text data can be information entered by the user before watching the video, or it can be the barrage information displayed on the page in the form of barrage during watching the video, or it can be the comment information entered after watching the video. For another example, the text data can be text information entered by the user using the application during the concert. In addition, the text data can be comment information obtained after purchasing the product, or it can be product information entered by the merchant to describe the product. For another example, in the scenario of news applications, the text data can be news text or comment information entered by the user after reading a certain news.

[0029] In the embodiment of the present invention, after authorization, the text data in the web page can be extracted by using the web page text extraction method.

[0030] Then, after the text data is extracted, the text data may be preprocessed. The preprocessing method of the text data includes: normalizing the format of the text data, such as unifying characters into lowercase characters, unifying text into simplified Chinese characters, replacing punctuation marks in the text data with spaces, etc. In addition, the preprocessing method of the text data also includes: filtering out noise in the text data, such as deleting web page links or web page publishing dates in the text data, etc.

[0031] Then, the preprocessed text data is split into individual text sentences according to punctuation marks. For example, the review text "The concert was great, I must watch the concert in Chengdu in December." can be directly divided into "The concert was great" and "I must watch the concert in Chengdu in December" according to punctuation marks. In this way, the candidate phrases can be made more readable.

[0032] Thereafter, candidate phrases can be generated using each text clause. Specifically, it can be determined whether the length of each text clause is less than a preset threshold. If the length of the text clause is less than the preset threshold, the text clause is determined as a candidate phrase. For example, when the preset threshold length is set to 6, the above-mentioned "The concert was great" can be directly determined as a candidate phrase.

[0033] For the case where the length of the text clause is greater than the preset threshold, such as the above-mentioned "I must watch the concert in Chengdu in December" , the text clause can be re-divided to obtain candidate phrases.

[0034] Specifically, the part-of-speech of each word segment in the text clause greater than the preset threshold can be determined first, and then the text clause can be divided using the part-of-speech of these word segments. That is to say, the text clause can be split into each word segment according to the part-of-speech. Preferably, HanLP can be used to perform word segmentation processing. HanLP is an NLP toolkit composed of a series of models and algorithms, which can include functions such as word segmentation, part-of-speech tagging, keyword extraction, and automatic summarization. By using HanLP to process the text data, each word segment constituting the text clause and its corresponding part-of-speech can be obtained. In addition, it should be noted that this application is not limited to this method, and any word segmentation tool that can be used to perform word segmentation processing can be applied here.

[0035] In addition, in order to prevent incorrect word segmentation, proper nouns / words in the comment content can be statistically obtained from a large number of samples in advance. In this process, useless word segments can be removed first. Useless word segments can include low-frequency word segments / characters and stop word segments. That is to say, low-frequency word segments / characters and stop word segments in the input text can be removed. Low-frequency word segments / characters can be words or characters with relatively low occurrence probabilities statistically obtained from a large number of text samples in advance. Stop word segments can be determined by looking up a stop word library, where the stop word library can be generated in advance based on predetermined rules. For example, words such as "de", "di", "de" can be included in the stop word library.

[0036] Finally, according to the part of speech of each segmentation, the text segment that meets the window size can be extracted from the text segment that is greater than the preset threshold value according to the window setting method as a candidate phrase. The window size can be preset by the technician according to the user's viewing preference. For example, the window size can be set to 5 and the segmentations in the window are complete in terms of part of speech. As described above, the sentence "I must watch the concert in Chengdu in December" can be divided into "I", "certain", "need", "live", "watch", "December", "Chengdu" and "concert" according to the part of speech. When the window size is set to 5, the phrases "I must", "must be live", "want to watch live", "live watch", "watch December", "Chengdu in December", and "Chengdu concert" can be extracted from the sentence. These phrases can be determined as candidate phrases. In this way, the candidate phrases can be made more in line with the user's reading habits.

[0037] According to an exemplary embodiment of the present application, after the candidate phrases are acquired in step S210, step S220 may be directly executed to determine whether each word segment in each candidate phrase belongs to at least one preset dictionary.

[0038] To better describe the process, please refer to Figure 3 To explain this in detail, Figure 3 A flowchart of steps for generating at least one predetermined dictionary according to an exemplary embodiment of the present application is shown.

[0039] In practice, the processing of text data is usually limited to a certain field, and the words in the special dictionary are words related to the set field corresponding to the special dictionary. Therefore, a special dictionary for the set field can be created in advance, and the set keywords can be determined from the results of the word segmentation processing through the special dictionary, and the word frequency statistics of the set keywords can be performed to perform text mining more targetedly.

[0040] In step S310, historical text data is collected. Specifically, a large amount of historical text data in a set field can be extracted using network text extraction technology. In implementation, the historical text data can also be preprocessed according to the preprocessing method mentioned above.

[0041] Then, in step S320, word segmentation is performed on the historical text data in the above manner to obtain high-frequency words.

[0042] In step S330, these high frequency words are annotated to generate a sentiment word dictionary and a subject word dictionary. The annotating operation can be performed manually or by artificial intelligence according to the needs.

[0043] In this process, the method can further optimize the generated sentiment word dictionary and main word dictionary based on these dictionaries, specifically including: introducing a variety of synonym generation methods based on the sentiment word dictionary and the main word dictionary, further generating sentiment word synonym dictionary and main word synonym dictionary, mainly including rule synonyms, case synonyms, word order reversal synonyms, meaning synonyms, Chinese and English synonyms, word vector synonyms, etc., wherein the rules are provided by domain experts, Chinese and English synonyms are generated by combining a third-party translation API and third-party encyclopedia knowledge, the synonyms under each synonym rule are iteratively optimized, and the final sentiment word synonym dictionary and main word synonym dictionary are generated by summarizing.

[0044] In the case where at least one preset dictionary has been generated in the above manner, each word segment in the candidate phrase can be compared with these preset dictionaries to determine that at least one word segment belongs to the candidate sentence of at least one preset dictionary. For example, "the concert is great" can be divided into "concert", "very" and "great", and then these three word segments are matched with the sentiment word dictionary and the subject word dictionary respectively.

[0045] Step S230, if it belongs to at least one preset dictionary, the corresponding candidate phrase is determined as a visual phrase. The visual phrase can be understood as a phrase that can be displayed on the screen as a word cloud. For example, based on determining that "concert" belongs to the main word dictionary and "good" belongs to the emotional word dictionary, "the concert is very good" to which it belongs can be determined as a visual phrase. In implementation, as long as there is a participle in the candidate phrase that belongs to at least one of the preset dictionaries mentioned above, the candidate phrase can be determined as a visual phrase.

[0046] As described above, by using a preset dictionary, a visualization phrase is determined from candidate phrases, so that the visualization phrase includes at least the comment subject word or the comment sentiment word, so that the semantics of the generated word cloud is more comprehensive and complete, and better reflects the user's true emotions.

[0047] As an example, the method can also directly perform word segmentation processing on the acquired text data, and extract multiple high-frequency word segments from the acquired word segments, and then perform clustering processing on these high-frequency word segments to obtain clustered visualized word segments. In other words, based on the existing word cloud generated according to words, these words can be clustered according to the synonym dictionary, and the word segment with the highest frequency in the cluster represents all the word segments of the cluster.

[0048] In summary, clustering is performed on the same or similar segmented words, so that the generated word cloud has more information and presents important comments more scientifically. Furthermore, a hybrid structure word cloud can be formed by using visualized phrases and visualized segmented words, so as to better show the real comment content to users.

[0049] In addition, the method can also generate a visual word cloud from the visual phrases and visual word segments in a preset manner. For example, after determining the visual phrases and visual word segments, the TF-IDF (term frequency–inverse document frequency) technology can be used to calculate the TF-IDF of each visual phrase and each visual word segment, and then the final displayed visual phrases and visual word segments are determined by the TF-IDF. In addition, the display method can also be determined according to the TF-IDF. For example, the higher the TF-IDF, the more centered the display position of the visual word segment, and the larger the displayed area.

[0050] TF-IDF is a statistical method used to evaluate the importance of a word to a document set or a document in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a document, but decreases in inverse proportion to the frequency of its appearance in the corpus. In this application, since the text data exists in the form of comments, when calculating TF-IDF, each comment can be regarded as a document to calculate TF and IDF.

[0051] In summary, the text data processing method of the exemplary embodiment of the present application can extract visual phrases from text data using a preset dictionary, so that the presented word cloud is more readable and more semantically rich. Further, the text data can be split using punctuation marks to obtain candidate phrases, so that the candidate phrases are more readable. Further, the text data can be windowed using parts of speech to obtain candidate phrases, so that the candidate phrases are more in line with the user's reading habits. Further, using a preset dictionary, visual phrases are determined from the candidate phrases, so that the visual phrases at least include comment subject words or comment sentiment words, so that the semantics of the generated word cloud are more comprehensive and complete, and the user's real emotions are better fed back. Further, clustering processing is performed on the same or similar segmented words, so that the generated word cloud has a larger amount of information and presents important comments more scientifically. Further, a hybrid structure word cloud can be formed using visual phrases and visual segmented words, so that the real comment content can be better displayed to the user.

[0052] As mentioned above, word clouds are used to visually highlight keywords that appear frequently in text data, forming a keyword cloud layer, thereby filtering out a large number of keywords that appear less frequently, so that users can quickly locate keywords that appear frequently. Figure 4 It can intuitively and quickly identify the audience's intuitive feelings after watching XX concert.

[0053] It can be seen intuitively from the word cloud that the word cloud contains not only visualized phrases, such as "feeling great at the scene" and "the singer's singing skills are great", but also visualized segmented words, such as "superb" and "happy". Through this mixed word cloud approach, users can more intuitively obtain richer information.

[0054] In addition, it should be noted that although Figure 4 The processing performed on the comment information of watching a concert according to the text data processing method of the exemplary embodiment of the present application is given, but it is not for limiting purposes. The text data processing method of the exemplary embodiment of the present application can also be applied to other scenarios for generating word clouds. For example, the barrage information during the video viewing can be processed according to the text data processing method of the exemplary embodiment of the present application, so that the word cloud generated by the comment information is displayed in the above manner. For another example, the collected product review information can be processed according to the text data processing method of the exemplary embodiment of the present application, so that the word cloud generated by the product review information is displayed in the above manner. For another example, the news text can be processed according to the text data processing method of the exemplary embodiment of the present application, so as to display the word cloud corresponding to the news text in the above manner.

[0055] In order to better illustrate this application, the following reference will be made to Figure 5 A text data processing method according to another exemplary embodiment of the present application is described. Figure 5 A flowchart of a text data processing method according to another exemplary embodiment of the present application is shown.

[0056] Reference Figure 5 , the text data processing method may include a generating module 510 and a text data processing module 520 .

[0057] The dictionary generation module 510 can generate various dictionaries in advance, so that the candidate phrases in the text data can be processed. Specifically, the historical text data can be collected, and then the historical texts can be segmented according to the above method, and then high-frequency words can be extracted from the segmented words. In this process, the high-frequency words can be labeled separately in combination with the part of speech of the high-frequency words to generate the comment sentiment word dictionary 6.1 and the comment subject word dictionary 6.2.

[0058] In addition, these dictionaries can be further optimized, including: introducing a variety of synonym generation methods based on the sentiment word dictionary and the main word dictionary, further generating sentiment word synonym dictionaries and the main word synonym dictionaries, mainly including rule synonyms, case synonyms, word order reversal synonyms, meaning synonyms, Chinese and English synonyms, word vector synonyms, etc., where the rules are provided by domain experts, Chinese and English synonyms are generated by combining third-party translation APIs and third-party encyclopedia knowledge, synonyms under each synonym rule are iteratively optimized, and the final sentiment word synonym dictionary and the main word synonym dictionary are generated by summarizing.

[0059] After extracting the comment information from the entire network, the text data processing module 520 may perform preprocessing on the text data, for example, converting the text data into traditional and simplified Chinese, converting full-width characters into half-width characters, removing emoticons, etc., and then perform sentence processing on the preprocessed text data, such as Figure 5 As shown, sentence processing can be implemented according to punctuation marks. Then, it is determined whether the length of each text sentence is less than a preset threshold; if the length of the text sentence is less than the preset threshold, the text sentence is determined as a candidate phrase (step 8).

[0060] If the length of the text sentence is greater than a preset threshold, the part of speech of each word in the text sentence greater than the preset threshold is determined (step 9), and then according to the part of speech of the word, the text sentence that meets the window size is extracted from the text sentence greater than the preset threshold in accordance with the window setting method as a candidate phrase (step 10).

[0061] Subsequently, it is determined whether each word segment in each candidate phrase belongs to at least one preset dictionary generated by the dictionary generation module 510 (step 13), and then the word segments belonging to the same preset dictionary are respectively calculated according to the TF-IDF method for each visualized phrase and each visualized word segment, and finally a visualized word cloud is generated (step 14). In order to more clearly understand the inventive concept of the exemplary embodiment of the present application, the following will refer to Figure 6 A block diagram of a text data processing device describing an exemplary embodiment of the present application. A person of ordinary skill in the art will understand that: Figure 6 The device in the embodiment only shows the components related to the exemplary embodiment, and the device also includes Figure 6 Common components other than those shown in .

[0062] Figure 6 A block diagram showing a text data processing device according to an exemplary embodiment of the present application. Figure 6At the hardware level, the device includes a processor, an internal bus and a computer-readable storage medium, wherein the computer-readable storage medium includes a volatile memory and a non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory and then runs it. Of course, in addition to the software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0063] Specifically, the processor performs the following operations: converting the acquired text data into candidate phrases in a predetermined manner; determining whether each word segment in each candidate phrase belongs to at least one preset dictionary; if it belongs to at least one preset dictionary, determining the corresponding candidate phrase as a visual phrase.

[0064] Optionally, the processor may implement steps to convert the acquired text data into candidate phrases in a predetermined manner, including: acquiring text data from a web page; splitting the text data into individual text sentences according to punctuation marks; and generating candidate phrases using the individual text sentences.

[0065] Optionally, the processor may implement the step of generating candidate phrases using the text sentences, including: determining whether the length of each text sentence is less than a preset threshold; if the length of the text sentence is less than the preset threshold, the text sentence is determined as a candidate phrase.

[0066] Optionally, the processor may further implement the steps of: if the length of the text sentence is greater than the preset threshold, determining the part of speech of each word in the text sentence greater than the preset threshold; and extracting text sentences that meet the window size from the text sentences greater than the preset threshold as candidate phrases according to the part of speech of each word.

[0067] Optionally, the processor may implement a step before splitting the text data into text sentences according to punctuation marks, including: performing preprocessing on the text data, and performing a splitting step on the preprocessed text data.

[0068] Optionally, the processor may implement steps to determine whether a word segment in each candidate phrase belongs to at least one preset dictionary, including: generating the at least one preset dictionary using historical text data; dividing the candidate phrase into word segments of different parts of speech according to the parts of speech of the candidate phrase; and determining whether each word segment belongs to the at least one preset dictionary.

[0069] Optionally, the processor may implement the step of generating the at least one preset dictionary using the historical text data, comprising: performing clustering processing on the words in the at least one preset dictionary to generate a synonym dictionary including different clusters.

[0070] Optionally, after the candidate phrase corresponding to the processor-implementable step is determined to be a visualized phrase, the method further includes: clustering each word segment in the visualized phrase into the same cluster word segment according to the at least one synonym dictionary, and generating clustered visualized word segmentations.

[0071] Optionally, the processor may further implement the steps of: extracting a plurality of high-frequency participles from the acquired text data; and clustering the plurality of high-frequency participles to acquire clustered visualized participles.

[0072] Optionally, the processor may further implement the step of: generating a visual word cloud from the visual phrases and the visual word segments in a preset manner.

[0073] Optionally, the at least one preset dictionary includes a sentiment word dictionary and a subject word dictionary.

[0074] In summary, according to the exemplary embodiment of the present application, the text data processing device can extract visual phrases from the text data using a preset dictionary, so that the presented word cloud is more readable and more semantically rich. Further, the text data can be split using punctuation marks to obtain candidate phrases, so that the candidate phrases are more readable. Further, the text data can be windowed using parts of speech to obtain candidate phrases, so that the candidate phrases are more in line with the user's reading habits. Further, using a preset dictionary, a visual phrase is determined from the candidate phrases, so that the visual phrase includes at least a comment subject word or a comment sentiment word, so that the semantics of the generated word cloud is more comprehensive and complete, and the user's real emotions are better fed back. Further, clustering processing is performed on the same or similar segmented words, so that the generated word cloud has a larger amount of information and presents important comments more scientifically. Further, a hybrid structure word cloud can be formed using visual phrases and visual segmented words, so that the real comment content can be better displayed to the user.

[0075] It should be noted that the execution subject of each step of the method provided in Example 1 can be the same device, or the method can be executed by different devices. For example, the execution subject of step 21 and step 22 can be device 1, and the execution subject of step 23 can be device 2; for another example, the execution subject of step 21 can be device 1, and the execution subject of step 22 and step 23 can be device 2; and so on.

[0076] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0080] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0081] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0082] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0083] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0084] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0085] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A text data processing method, the method comprising: Convert the acquired text data into candidate phrases in a predetermined manner; Determining whether a word in each candidate phrase belongs to at least one preset dictionary; If the candidate phrase belongs to the at least one preset dictionary, determining the corresponding candidate phrase as a visual phrase; The step of determining whether a word segment in each candidate phrase belongs to at least one preset dictionary comprises: generating the at least one preset dictionary using historical text data; dividing the candidate phrase into word segments of different parts of speech according to the part of speech of the candidate phrase; and determining whether each word segment belongs to the at least one preset dictionary; After generating the at least one preset dictionary using the historical text data, the method further comprises: based on the at least one preset dictionary, introducing a corresponding synonym dictionary to generate the corresponding at least one preset synonym dictionary; After determining the corresponding candidate phrase as a visualized phrase, the method further includes: clustering each word segment in the visualized phrase according to the at least one preset synonym dictionary to generate clustered visualized word segments; Generate a visual word cloud using the visual phrases and visual word segments in a preset manner; The at least one preset dictionary includes a sentiment word dictionary and a subject word dictionary, and the corresponding candidate phrases include sentiment words and subject words.

2. The method of claim 1, wherein: Converting the acquired text data into candidate phrases in a predetermined manner includes: Get text data from web pages; Splitting the text data into text sentences according to punctuation marks; The text sentences are used to generate candidate phrases.

3. The method of claim 2, wherein: Generating candidate phrases using the text sentence segmentation includes: Determine whether the length of each text sentence is less than a preset threshold; If the length of the text sentence is less than a preset threshold, the text sentence is determined to be a candidate phrase.

4. The method of claim 3, further comprising: If the length of the text sentence is greater than the preset threshold, determining the part of speech of the participle in the text sentence greater than the preset threshold; According to the part of speech of the word segment, text sentences satisfying the window size are extracted from the text sentences larger than the preset threshold in a window setting manner as candidate phrases.

5. The method of claim 2, wherein: Before the text data is split into text sentences according to punctuation marks, the following steps are included: The text data is preprocessed and then the preprocessed text data is subjected to a splitting step.

6. The method of claim 1, further comprising: Extract multiple high-frequency words from the acquired text data; The multiple high-frequency segmented words are clustered to obtain clustered visualized segmented words.

7. A text data processing device, the device comprising: processor; as well as A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any one of claims 1 to 6.

8. A computer-readable storage medium having computer instructions stored thereon, wherein the instructions are executed to implement the method of any one of claims 1 to 6.

Images