Text data processing method, device, electronic device and storage medium

Abstract

The embodiments of the present disclosure disclose a text data processing method and device, an electronic device and a storage medium, wherein the method comprises: obtaining text data to be processed; segmenting the text data to be processed to obtain a plurality of text data segments; classifying the plurality of text data segments according to the classification method corresponding to the data item to which the text data to be processed belongs, and obtaining classification processing results of the plurality of text data segments, wherein the classification methods corresponding to at least two data items are different; performing text extraction processing on the plurality of text data segments according to the text extraction method corresponding to the classification processing results of the plurality of text data segments, and obtaining valid text segments corresponding to the plurality of text data segments; using the valid text segments corresponding to the plurality of text data segments as valid text data of the text data to be processed. The present disclosure is conducive to efficiently and accurately obtaining text data that meets the needs of subsequent processes.

Description

Technical Field

[0001] The present disclosure relates to computer technology, and in particular to a text data processing method, a text data processing device, an electronic device, and a storage medium. Background Art

[0002] Since the collected text data often has many uncertain factors in terms of format and content, it is usually necessary to process the collected text data before it can be provided to the subsequent processes. For example, compared with the information transmission method of text, multimedia (such as video) is conducive to enabling people to obtain information more easily and quickly. For example, a few seconds or tens of seconds of multimedia can enable people to understand the general situation of an object. The general situation of the object can be the performance of a commodity, the content of a news item, or the historical anecdotes of a place. In the process of using the collected text data to generate multimedia for multiple objects, it is often necessary to first process the collected text data, and then use the processed text data to generate multimedia for the object.

[0003] How to efficiently and accurately provide text data that meets the needs of subsequent processes is a technical issue worthy of attention. Summary of the invention

[0004] The disclosed embodiment provides a technical solution for text data processing to efficiently and accurately obtain valid text data of text data to be processed.

[0005] According to one aspect of an embodiment of the present disclosure, a text data processing method is provided, comprising: acquiring text data to be processed; segmenting the text data to be processed to obtain a plurality of text data segments; classifying the plurality of text data segments according to a classification method corresponding to a data item to which the text data to be processed belongs, to obtain classification results of the plurality of text data segments, wherein the classification methods corresponding to at least two data items are different; performing text extraction on the plurality of text data segments according to a text extraction method corresponding to the classification results of the plurality of text data segments, to obtain valid text segments corresponding to the plurality of text data segments; using the valid text segments corresponding to the plurality of text data segments as valid text data of the text data to be processed.

[0006] Optionally, in the method of any embodiment of the present disclosure, obtaining the text data to be processed includes: obtaining long text data, wherein the long text data includes: text data whose byte length satisfies a predetermined byte length requirement; determining a dirty word set corresponding to a data item to which the long text data belongs, wherein the dirty word sets corresponding to at least two data items are different; performing data cleaning on the long text data according to the dirty word set corresponding to the data item to which the long text data belongs, and using the result of the data cleaning as the text data to be processed.

[0007] Optionally, in the method of any embodiment of the present disclosure, the multiple text data segments are classified and processed respectively according to the classification method corresponding to the data item to which the text data to be processed belongs, and the classification processing results of each of the multiple text data segments are obtained, including: for any text data segment, according to the priority of multiple classification conditions contained in the classification method corresponding to the data item to which the text data to be processed belongs, determining the classification condition that the text data segment satisfies for the first time, and using the classification condition that is satisfied for the first time as the classification processing result of the text data segment.

[0008] Optionally, in the method of any embodiment of the present disclosure, when the data item to which the text data to be processed belongs is the first data item, the multiple classification conditions contained in the classification method corresponding to the first data item include at least one of the following, and the priority of the multiple classification conditions is arranged in the following order from high to low: whether the text data segment belongs to a legal sentence containing predetermined legal terms; whether the text data segment belongs to an introductory sentence containing a colon; whether the text data segment belongs to a gerund sentence containing a colon; whether the text data segment belongs to a chemical formula sentence containing predetermined characters based on a chemical formula; whether the text data segment belongs to a sentence containing predetermined auxiliary words; whether the text data segment belongs to a sentence containing predetermined punctuation marks.

[0009] Optionally, in the method of any embodiment of the present disclosure, the text extraction method corresponding to the classification processing results of each of the multiple text data segments is used to perform text extraction processing on the multiple text data segments respectively to obtain valid text segments corresponding to each of the multiple text data segments, including at least one of the following: no text extraction processing is performed on the legal sentence, and the valid text segment of the legal sentence is empty; the sentence after the colon is extracted from the introductory sentence, and the sentence after the colon is used as the valid text segment of the introductory sentence; based on the verbs and nouns in the gerund sentence, a valid text segment of noun plus predetermined particle plus verb is generated; a predetermined number of chemical formulas in the chemical formula sentence are used as valid text segments; based on the verbs and nouns in the sentence containing the predetermined particle, a valid text segment of noun plus predetermined particle plus verb is generated; based on the predetermined punctuation marks contained in the text data segment, the text data segment is segmented, and each segment obtained by the segmentation process is used as a valid text segment.

[0010] Optionally, in the method of any embodiment of the present disclosure, when the data item to which the text data to be processed belongs is a second data item, the multiple classification conditions contained in the classification method corresponding to the second data item include at least one of the following, and the priorities of the multiple classification conditions are arranged in the following order from high to low: whether the text data segment belongs to a sentence containing predetermined words based on affiliation and valid nouns in a predetermined valid noun set; whether the text data segment belongs to a sentence containing valid nouns in a predetermined valid noun set.

[0011] Optionally, in the method of any embodiment of the present disclosure, the text extraction processing is performed on the multiple text data segments respectively according to the text extraction method corresponding to the classification processing results of each of the multiple text data segments to obtain valid text segments corresponding to each of the multiple text data segments, including at least one of the following: taking a text data segment containing predetermined words based on affiliation and valid nouns in a predetermined valid noun set as a valid text segment; taking a text data segment containing valid nouns in a predetermined valid noun set as a valid text segment.

[0012] Optionally, in the method of any embodiment of the present disclosure, the method further includes: performing part-of-speech segmentation processing on multiple text data to be processed belonging to the second data item to obtain the part-of-speech of each word contained in each of the multiple text data to be processed; performing word frequency statistics on all words whose part of speech is noun to obtain the word frequency statistics results of each word whose part of speech is noun; selecting nouns whose word frequency statistics results meet predetermined word frequency conditions from all words whose part of speech is noun to form the valid noun set.

[0013] Optionally, in the method of any embodiment of the present disclosure, the data item to which the text data to be processed belongs is used as the information display dimension of the object to be processed, and the valid text data of the text data to be processed is used as the content of the information display dimension of the object to be processed; the method also includes: generating a multimedia unit in the multimedia file of the object to be processed according to the content of the information display dimension of the object to be processed.

[0014] Optionally, in the method of any embodiment of the present disclosure, generating a multimedia unit in the multimedia file of the object to be processed according to the content of the information display dimension of the object to be processed includes: acquiring all information display dimensions of multiple objects to be processed; for at least one information display dimension among all the information display dimensions, performing dimension commonality detection on the at least one information display dimension according to the content of the at least one information display dimension of the multiple objects to be processed, so as to determine whether the at least one information display dimension is respectively an information display dimension with dimension commonality; for any information display dimension with dimension commonality, updating the content of the information display dimension using the commonality label of the information display dimension, wherein any information display dimension with dimension commonality has at least one commonality label; generating multiple multimedia units using all the information display dimensions and their contents, wherein one information display dimension corresponds to different multimedia units according to its different commonality labels, and all the information display dimensions and their contents include: information display dimensions with dimension commonality and their updated contents; for any object to be processed, selecting a multimedia unit belonging to the object to be processed from the multimedia units according to all the information display dimensions and their contents contained in the object to be processed, and synthesizing all the selected multimedia units into the multimedia file of the object to be processed.

[0015] Another aspect of the disclosed embodiment provides a text data processing device, including: a text data acquisition module for acquiring text data to be processed; a segmentation processing module for segmenting the text data to be processed to obtain a plurality of text data segments; a classification processing module for classifying the plurality of text data segments according to the classification methods corresponding to the data items to which the text data to be processed belongs, to obtain classification processing results of the plurality of text data segments, wherein the classification methods corresponding to at least two data items are different; an extraction processing module for performing text extraction processing on the plurality of text data segments according to the text extraction methods corresponding to the classification processing results of the plurality of text data segments, to obtain valid text segments corresponding to the plurality of text data segments; and a valid text data module for using the valid text segments corresponding to the plurality of text data segments as valid text data of the text data to be processed.

[0016] Optionally, in the device of any embodiment of the present disclosure, the data item to which the text data to be processed belongs is used as the information display dimension of the object to be processed, and the valid text data of the text data to be processed is used as the content of the information display dimension of the object to be processed, and the device further includes: a display dimension acquisition module, used to acquire all information display dimensions of multiple objects to be processed; a dimension commonality determination module, used to perform dimension commonality detection on at least one information display dimension among all the information display dimensions according to the content of the at least one information display dimension of the multiple objects to be processed, so as to determine whether the at least one information display dimension is an information display dimension with dimension commonality; a dimension content update module, used to update the dimension content of any information display dimension with dimension commonality. , using the common label of the information display dimension to update the content of the information display dimension, wherein any information display dimension with dimension commonality has at least one common label; generating a media unit module, used to generate multiple multimedia units using all the information display dimensions and their contents, wherein one information display dimension corresponds to different multimedia units according to its different common labels, and all the information display dimensions and their contents include: information display dimensions with dimension commonality and their updated contents; synthesizing a media file module, used to select, for any object to be processed, the multimedia unit belonging to the object to be processed from the multimedia units according to all the information display dimensions and their contents contained in the object to be processed, and synthesize all the selected multimedia units into a multimedia file of the object to be processed.

[0017] Another aspect of the embodiments of the present disclosure provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the text data processing method described in any of the above embodiments of the present disclosure is implemented.

[0018] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the text data processing method described in any of the above embodiments of the present disclosure is implemented.

[0019] Based on the text data processing method and device, electronic device and storage medium provided by the above-mentioned embodiments of the present disclosure, by segmenting the multiple text data segments obtained from the text data to be processed, each text data segment is classified and processed based on the classification method corresponding to the corresponding data item, so that each text data segment in the text data to be processed can be subjected to text extraction processing using the corresponding text extraction method, thereby facilitating accurate acquisition of the valid text segment of each text data segment, and by taking the valid text segments of all text data segments as the valid text data of the text data to be processed, even when the text data to be processed is long text data, the present disclosure can accurately obtain the valid text data of the text data to be processed. It can be seen from this that the embodiments of the present disclosure are conducive to efficiently and accurately obtaining text data that meets the needs of subsequent processes.

[0020] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0023] Figure 1 The present invention is a flowchart of a text data processing method according to an embodiment of the present invention.

[0024] Figure 2 The present invention is a flowchart of an embodiment of a process for obtaining text data to be processed according to an embodiment of the present invention.

[0025] Figure 3 The present invention is a flowchart of an embodiment of a process for setting a valid noun set according to an embodiment of the present disclosure.

[0026] Figure 4 The present invention is a flowchart of an embodiment of using valid text data to generate a multimedia file for an object to be processed.

[0027] Figure 5 The present invention is a flowchart of an embodiment of generating multiple multimedia units according to an embodiment of the present disclosure.

[0028] Figure 6 The present invention is a flowchart of an embodiment of generating a multimedia unit of an information display dimension according to an embodiment of the present disclosure.

[0029] Figure 7 The present invention is a flowchart of an embodiment of synthesizing a multimedia file into an object to be processed.

[0030] Figure 8It is a structural schematic diagram of an embodiment of a text data processing device disclosed in the present invention.

[0031] Fig. 9 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0032] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0033] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0034] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0035] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0036] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0037] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0038] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0039] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0040] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0041] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0042] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0043] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0044] Figure 1 FIG. 1 is a flowchart of an embodiment of the text data processing method disclosed in the present invention. Figure 1 As shown, the text data processing method of this embodiment includes the following steps:

[0045] S100: Obtain text data to be processed.

[0046] The text data to be processed in the present disclosure may refer to text data from which valid text data needs to be extracted. The text data to be processed in the present disclosure may be text data crawled from the network. In one example, the text data to be processed in the present disclosure may be long text data. The long text data may refer to text data whose byte length meets the predetermined byte length requirement, for example, the long text data may be text data whose byte length is greater than the predetermined byte length. The text data to be processed in the present disclosure may include: foreign languages ​​(such as English, Greek letters and Latin letters, etc.), symbols (such as punctuation marks, mathematical operation symbols and special symbols, etc.), numbers and Chinese characters in various forms. The present disclosure does not limit the specific content of the text data to be processed.

[0047] S101, segmenting the text data to be processed to obtain multiple text data segments.

[0048] The present disclosure can segment the text data to be processed according to the separators contained in the text data to be processed, and any text data segment obtained by the segmentation process can be called a sub-sentence. A sub-sentence can be in the form of one or more short sentences, one or more phrases, or a complete sentence.

[0049] S102 , classifying and processing the plurality of text data segments respectively according to the classification methods corresponding to the data items to which the text data to be processed belongs, and obtaining classification processing results for the plurality of text data segments respectively.

[0050] In the case where the text data to be processed is stored in a data warehouse, the data item to which the text data to be processed in the present disclosure belongs may refer to the field name to which the text data to be processed belongs. The specific form of expression of the data item to which the text data to be processed belongs is usually related to the application to which the technology of the present disclosure is applicable. For example, in a company query application, the data item to which the text data to be processed belongs may be "company business scope" or "company project brand", etc. For another example, in a commodity retail application, the data item to which the text data to be processed belongs may be "commodity details" or "commodity usage method" or "commodity ingredients", etc.

[0051] The classification method in the present disclosure may refer to information used to represent a specific operation of classifying a text data segment, that is, the classification method is used to define how to classify a text data segment. In one example, a classification method may include a plurality of pre-set classification conditions, that is, the present disclosure may implement classification of a text data segment by determining the classification conditions satisfied by the text data segment.

[0052] Each data item in the present disclosure corresponds to a classification method. In general, at least two data items among all the data items correspond to different classification methods. For example, when there are three data items, if the first data item corresponds to the first classification method, the second data item corresponds to the second classification method, and the third data item corresponds to the third classification method, then the three classification methods are all different. For another example, when there are three data items, if the first data item and the second data item both correspond to the first classification method, and the third data item corresponds to the second classification method, then the two classification methods are different. However, the present disclosure does not exclude the situation where all data items correspond to the same classification method.

[0053] The purpose of the classification processing in the present disclosure at least includes: determining the text extraction method of the text data segment. That is to say, the present disclosure can determine the category to which the text data segment belongs by classifying the text data segment. The category can be called an extraction category, and each extraction category corresponds to a specific text extraction method.

[0054] S103 . Perform text extraction processing on the multiple text data segments respectively according to the text extraction methods corresponding to the classification processing results of the multiple text data segments, to obtain valid text segments corresponding to the multiple text data segments respectively.

[0055] The text extraction method in the present disclosure may refer to information used to represent an implementation method for extracting valid text data from a text data segment. In one example, when a classification method in the present disclosure includes multiple classification conditions, the present disclosure may set a text extraction method for each classification condition, and a text extraction method corresponds to an extraction category, so that when it is determined that a text data segment meets a classification condition, the corresponding text extraction process can be performed on the text data segment according to the text extraction method corresponding to the classification condition, and the result of the text extraction process is the valid text segment corresponding to the text data segment. The valid text segment in the present disclosure may be a short sentence or a phrase, etc.

[0056] S104: Use the valid text segments corresponding to the multiple text data segments as the valid text data of the text data to be processed.

[0057] The present disclosure can arrange all valid text segments according to the arrangement order of multiple text data segments in the text data to be processed, thereby forming valid text data of the text data to be processed. The valid text data of the text data to be processed in the present disclosure can be considered as the normalized text data of the text data to be processed. In one example, the valid text data in the present disclosure can refer to text data that meets the standard requirements of the subsequent process for text data. The subsequent process can refer to a process that needs to use text data. For example, in an application scenario of generating a formatted table, the subsequent process can be a process of generating a table using valid text data; for another example, in an application scenario of generating multimedia files using text, the subsequent process can be a process of generating multimedia files using valid text data. The present disclosure does not limit the specific form of expression of the subsequent process.

[0058] The present disclosure obtains multiple text data segments by segmenting the text data to be processed, and classifies each text data segment based on the classification method corresponding to the corresponding data item, so that each text data segment in the text data to be processed can be processed by using the corresponding text extraction method, which is conducive to accurately obtaining the valid text segment of each text data segment; by taking the valid text segments of all text data segments as the valid text data of the text data to be processed, even when the text data to be processed is long text data, the present disclosure can accurately obtain the valid text data of the text data to be processed. It can be seen that the embodiment of the present disclosure is conducive to efficiently and accurately obtaining text data that meets the needs of subsequent processes.

[0059] In an optional example, the text data to be processed in the present disclosure is text data that has been preprocessed (such as dirty word cleaning). That is, the present disclosure can obtain the text data to be processed by preprocessing the text data such as dirty word cleaning. Figure 2 shown.

[0060] Figure 2 In S200, long text data is obtained.

[0061] Optionally, the long text data in the present disclosure may be text data crawled from the Internet, or may be text data uploaded by a user, etc. The byte length of the long text data should meet a predetermined byte length requirement, for example, the byte length of the long text data should not be less than a predetermined byte length. The long text data in the present disclosure may also be referred to as original text data or initial text data, etc.

[0062] S201: Determine a dirty word set corresponding to a data item to which long text data belongs.

[0063] Optionally, the dirty word set in the present disclosure may refer to a set formed by words (such as characters or words, etc.) that are not desired to appear in the text data to be processed. The dirty word set may also be referred to as a dirty word word list. Each data item in the present disclosure corresponds to a dirty word set. For example, the dirty word sets corresponding to any two data items in all data items are different. For another example, the dirty word sets corresponding to at least two data items in all data items are different.

[0064] Optionally, the words (which may be called dirty words) included in each dirty word set may be set according to the actual needs of the corresponding data item. In one example, for a data item, a predetermined word (such as a predetermined character or a predetermined word, etc.) may belong to the dirty word set corresponding to the data item, but the predetermined word may not belong to the dirty word set corresponding to another data item. As a more specific example, for the data item "company's project brand", "α" and "β" both belong to the dirty word set corresponding to the data item, but "α" and "β" usually do not belong to the dirty word set corresponding to the data item "company's business scope".

[0065] Optionally, an example of obtaining a dirty word set corresponding to a data item in the present disclosure may be:

[0066] First, all long text data belonging to the data item are formed into a text data set;

[0067] Secondly, all the long text data in the text data set are segmented to obtain all the words (the words here can be characters or phrases, etc.) contained in each long text data;

[0068] Afterwards, word frequency statistics are performed on the result of the word segmentation processing, and the dirty word set of the data item is determined according to the result of the word frequency statistics. For example, words with a word frequency less than a predetermined word frequency can be put into the dirty word set.

[0069] In addition, if the dirty word set corresponding to the data item has clear characteristics, the present disclosure can also put the corresponding words in the result of the word segmentation processing into the dirty word set based on the characteristics. For example, if a data item does not usually include characters formed by other than Chinese and English, the present disclosure can first put all non-Chinese and non-English words (such as Greek characters such as "α", "β", and mathematical operation symbols such as "+", "-") in the result of the word segmentation processing into the dirty word set, and then perform word frequency statistics processing on the remaining words after filtering out the non-Chinese and non-English words, and continue to determine the words in the dirty word set based on the results of the word frequency statistics processing. The present disclosure can use the above method to obtain the dirty word sets corresponding to all data items.

[0070] S202: Perform data cleaning on the long text data according to the dirty word set corresponding to the data item to which the long text data belongs, and use the result of the data cleaning as the text data to be processed.

[0071] Optionally, for any long text data, the present disclosure can determine whether the long text data contains words in the dirty word set corresponding to the data item to which the long text data belongs. If the long text data does not contain words in the dirty word set corresponding to the data item to which the long text data belongs, the long text data can be directly used as the text data to be processed. If the long text data contains words in the dirty word set corresponding to the data item to which the long text data belongs, such words should be deleted from the long text data, thereby obtaining the text data to be processed.

[0072] The present disclosure sets a dirty word set for each data item and uses the dirty word set corresponding to the data item to which the long text data belongs to perform data cleaning on the long text data, which is conducive to flexible and accurate data cleaning. In some applications, it is helpful to avoid the influence of inappropriate words on subsequent text data processing.

[0073] In an optional example, the present disclosure can segment the text data to be processed based on delimiters, thereby obtaining multiple text data segments. More specifically, for any text to be processed, the present disclosure can use the delimiter set corresponding to the data item to which the text data to be processed belongs to perform sentence segmentation on the text data to be processed, and obtain all sub-sentences contained in the text to be processed, each of which is a text data segment.

[0074] Optionally, each data item in the present disclosure corresponds to a delimiter set. In one example, any two data items in all data items correspond to different delimiter sets. In another example, at least two data items in all data items correspond to different delimiter sets.

[0075] Optionally, the delimiters contained in each delimiter set can be set according to the actual needs of the corresponding data item. In one example, for a data item, a predetermined delimiter (such as a comma, etc.) belongs to the delimiter set corresponding to the data item, but the predetermined delimiter does not belong to the delimiter set corresponding to another data item. In a more specific example, for the data item "company's project brand", the comma (including full-width commas and half-width commas) usually belongs to the delimiter set corresponding to the data item, but the comma usually does not belong to the delimiter set corresponding to the data item "company's business scope".

[0076] Optionally, for any text data to be processed, the present disclosure can start from the starting position in the text data to be processed to find the position of the first delimiter in the delimiter set corresponding to the data item to which the text data to be processed belongs, and use the content between the starting position and the found position as the first sub-statement, and then start from the currently found position to find the position of the second delimiter in the delimiter set corresponding to the data item to which the text data to be processed belongs, and use the content between the first found position and the second found position as the second sub-statement, and so on, until the last sub-statement of the text data to be processed is obtained.

[0077] The present disclosure sets a separator set for each data item respectively, and uses the separator set corresponding to the data item to which the text data to be processed belongs to perform segmentation processing on the text data to be processed, which is beneficial for flexible and accurate segmentation processing on the text data to be processed belonging to different data items. In some applications, it is beneficial to avoid the influence of inappropriate segmentation processing on the subsequent text data processing of the present disclosure.

[0078] In an optional example, the present disclosure can classify and process text data segments using at least one pre-set classification condition. In one example, for any text data segment in the text data to be processed, the present disclosure can determine the classification condition that the text data segment first meets based on the priority of all classification conditions contained in the classification method corresponding to the data item to which the text data to be processed belongs, and use the classification condition that is first met as the classification processing result of the text data segment.

[0079] Optionally, each data item in the present disclosure corresponds to a classification method, and each classification method generally includes multiple classification conditions. In one example, all classification conditions contained in the classification methods corresponding to any two data items are different. In another example, the classification conditions contained in the classification methods corresponding to any two data items are partially the same and partially different. For example, a classification condition belongs to the classification method corresponding to one data item, but the classification condition does not belong to the classification method corresponding to another data item; while the other classification condition belongs to the classification methods corresponding to both data items. All classification conditions contained in each classification method can be set according to the actual needs of the corresponding data items.

[0080] Optionally, for any text data segment in the text data to be processed, the present disclosure may sort all classification conditions contained in the classification method corresponding to the data item to which the text data to be processed belongs in order of priority from high to low, and first determine whether the text data segment satisfies the classification condition with the highest priority. If it satisfies the classification condition with the highest priority, it is determined that the text data segment should use the text extraction method corresponding to the classification condition with the highest priority for text extraction processing; otherwise, it is determined whether the text data segment satisfies the classification condition with the second highest priority. If it satisfies the classification condition with the second highest priority, it is determined that the text data segment should use the text extraction method corresponding to the classification condition with the second highest priority for text extraction processing, and so on, until the classification condition that the text data segment satisfies for the first time is determined.

[0081] The present invention sets a classification method for each data item and sets priorities for all classification conditions contained in the classification method, so that when the text data segment is classified, it is helpful to determine the appropriate classification processing result for the text data segment, and further helps to ensure the accuracy of the text extraction processing result of the text data segment.

[0082] In an optional example, when the data item to which the text data to be processed belongs is the first data item (such as "company business scope", etc.), the multiple classification conditions included in the classification method corresponding to the first data item of the present disclosure may include at least one of the following 6 classification conditions, and the following 6 classification conditions are arranged in descending order of priority:

[0083] Classification condition a: whether the text data segment belongs to a legal sentence containing predetermined legal terms. The predetermined legal terms may be "laws", "regulations" and "rules", etc. The present disclosure may perform word segmentation processing on the text data segment (such as using the stammering word segmentation to perform word segmentation processing on the text data segment), thereby obtaining all the words included in the text data segment, and determining whether there are predetermined legal terms among all the words included in the text segment of the file. If there are predetermined legal terms, it is determined that the text data segment belongs to a legal sentence containing predetermined legal terms, and the classification processing process for the text data segment ends; otherwise, the text data segment continues to be classified using classification condition b.

[0084] Classification condition b, whether the text data segment belongs to an introductory sentence containing a colon. The colon can be a full-width colon (such as the colon in Chinese punctuation marks) or a half-width colon (such as the colon in English punctuation marks); the introductory sentence containing a colon in the present disclosure may refer to a sentence in which the content after the colon is used to introduce and explain the content before the colon in detail. An example of an introductory sentence containing a colon may be, "The scope includes: auxiliary, work clothes, anti-static clothing". In the process of word segmentation processing of the text data segment, the present disclosure can obtain the parts of speech of all the words contained in the text data segment in addition to obtaining all the words contained in the text data segment. After that, the present disclosure can determine whether there are verbs before and after the colon. If there are no verbs, it can be determined that the text data segment belongs to an introductory sentence containing a colon, and the classification processing process for the text data segment ends; if there are verbs, the text data segment can continue to be classified using classification condition c.

[0085] Classification condition c, whether the text data segment belongs to a gerund sentence containing a colon. The colon can be a full-width colon (such as a colon in Chinese punctuation marks) or a half-width colon (such as a colon in English punctuation marks); the gerund sentence containing a colon in the present disclosure may refer to a sentence including a verb before the colon and a noun after the colon. An example of a gerund sentence containing a colon may be: "Wholesale, retail: fruits, vegetables". The present disclosure can determine whether a text data segment contains a verb before the colon and a noun after the colon. If a text data segment contains a verb before the colon and a noun after the colon, it can be determined that the text data segment belongs to a gerund sentence containing a colon, and the classification process for the text data segment ends; otherwise, the classification condition d can be used to continue to classify the text data segment.

[0086] Classification condition d: whether the text data segment belongs to a chemical formula sentence containing predetermined characters based on the chemical formula. The predetermined characters based on the chemical formula in the present disclosure may refer to characters that are usually contained in the chemical formula when the chemical formula is expressed in words, such as numbers, hyphens, and commas. An example of a chemical formula sentence may be "1,2-dichloroethane, n-octane, 2-chloropropane". The present disclosure may determine whether the text data segment contains predetermined characters based on the chemical formula. If the text data segment contains predetermined characters based on the chemical formula, it may be determined that the text data segment belongs to a chemical formula sentence containing predetermined characters based on the chemical formula, and the classification process for the text data segment ends; otherwise, the classification condition e may be used to continue to classify the text data segment.

[0087] Classification condition e: whether the text data segment belongs to a sentence containing a predetermined auxiliary word. The predetermined auxiliary word in the present disclosure may be "的". An example of a sentence containing a predetermined auxiliary word in the present disclosure may be "the development and sales of computer software and computer hardware". The present disclosure can determine whether the text data segment contains a predetermined auxiliary word. If the text data segment contains a predetermined auxiliary word, it can be determined that the text data segment belongs to a sentence containing a predetermined auxiliary word, and the classification process for the text data segment ends; otherwise, the classification process for the text data segment can continue using classification condition f.

[0088] Classification condition f: whether the text data segment belongs to a sentence containing a predetermined punctuation mark. The predetermined punctuation marks in the present disclosure may be a comma, a semicolon, etc. The present disclosure can determine whether the text data segment contains a predetermined punctuation mark. If the text data segment contains a predetermined punctuation mark, it can be determined that the text data segment belongs to a sentence containing a predetermined punctuation mark, and the classification process for the text data segment ends.

[0089] If a text data segment belongs to a legal sentence containing a predetermined legal term, the present disclosure can mark the text data segment as a legal sentence or as a text data segment that does not require text extraction processing, so that the text extraction processing operation is not performed on the text data segment subsequently, that is, the effective text segment corresponding to the text data segment is empty.

[0090] If a text data segment belongs to an introductory sentence containing a colon, the present disclosure can extract the sentence after the colon from the text data segment and use the sentence after the colon as the effective text segment of the text data segment. Continuing the previous example, if the text data segment is "The scope includes: auxiliary, work clothes, anti-static clothing", the effective text segment extracted by the present disclosure from this text data segment is "auxiliary, work clothes, anti-static clothing".

[0091] If a text data segment belongs to a verb-noun sentence containing a colon, the present disclosure can generate an effective text segment of noun + predetermined auxiliary word + verb according to the verb and noun in the text data segment, that is, the effective text segment is in the form of: noun + predetermined auxiliary word + verb. The predetermined auxiliary word therein may be "的". Continuing the previous example, if the text data segment is "Wholesale, retail: fruits, vegetables", the effective text segments extracted by the present disclosure from this text data segment are "the wholesale of fruits", "the retail of fruits", "the wholesale of vegetables", and "the retail of vegetables".

[0092] If a text data segment belongs to a chemical formula sentence, the present disclosure may take a predetermined number of chemical formulas in the text data segment as valid text segments. The predetermined number may be 2 or 3, etc. That is, the present disclosure may take part of the chemical formulas in the text data segment as valid text segments extracted. Continuing with the previous example, if the text data segment is "1,2-dichloroethane, n-octane, 2-chloropropane", the valid text segments extracted from the text data segment by the present disclosure are "1,2-dichloroethane" and "n-octane".

[0093] If a text data segment belongs to a sentence containing a predetermined particle, the present disclosure generates a valid text segment of noun plus predetermined particle plus verb according to the verbs and nouns in the text data segment, that is, the valid text segment can be expressed in the form of: noun + predetermined particle + verb. Continuing with the previous example, if a text data segment is "Development and sales of computer software and computer hardware", the valid text segments extracted from the text data segment by the present disclosure can be "Development of computer software", "Sales of computer software", "Development of computer hardware" and "Sales of computer hardware".

[0094] If a text data segment belongs to a sentence containing predetermined punctuation marks, the present disclosure can segment the text data segment, and use each segment obtained by the segmentation process as a valid text segment of the text data segment. In one example, the present disclosure can segment the text data segment according to the predetermined punctuation marks (such as commas and semicolons, etc.) contained in the text data segment.

[0095] If a text data segment does not belong to any of the above statements, the present disclosure may directly regard the text data segment as a valid text segment, or may regard the valid text segment of the text data segment as empty.

[0096] In an optional example, when the data item to which the text data to be processed belongs is a second data item (such as "the company's project brand", etc.), the multiple classification conditions included in the classification method corresponding to the second data item of the present disclosure may include at least one of the following two classification conditions, and the following two classification conditions are arranged in order of priority from high to low:

[0097] Classification condition A: whether the text data segment belongs to a sentence containing a predetermined word based on affiliation and a valid noun in a predetermined valid noun set. The predetermined word based on affiliation in the present disclosure may refer to a word used to indicate the affiliation between two parts. For example, the predetermined word based on affiliation may be "is", "is", "belongs to" or "belongs to", etc. The predetermined valid noun set in the present disclosure usually contains multiple valid nouns. An example of a sentence containing a predetermined word based on affiliation and a valid noun in a predetermined valid noun set may be: "Toy Superman is a children's toy sharing and rental platform". Another example of a sentence containing a predetermined word based on affiliation and a valid noun in a predetermined valid noun set may be: "Toy Superman is a platform mainly for infants and young children aged 0-6 years old and people", and another example may be: "and it is a children's toy sharing and rental platform". If the text data segment includes a predetermined word based on affiliation and any valid noun in a predetermined valid noun set, the present disclosure can determine that the text data segment belongs to a sentence containing a predetermined word based on affiliation and a valid noun in a predetermined valid noun set, and the classification processing process for the text data segment ends; otherwise, the text data segment can continue to be classified using classification condition B.

[0098] Classification condition B: whether the text data segment belongs to a sentence containing a valid noun in a predetermined valid noun set. An example of a sentence containing a valid noun in a predetermined valid noun set may be: "mainly for infants and young children aged 0-6 years old". If the text data segment includes any valid noun in the predetermined valid noun set, it can be determined that the text data segment belongs to a sentence containing a valid noun in the predetermined valid noun set, and the classification process for the text data segment ends.

[0099] If a text data segment belongs to a sentence containing a predetermined term based on affiliation and a valid noun in a predetermined valid noun set, the present disclosure can directly treat the text data segment as a valid text segment, and the text data segment can be considered as a canonical valid clause of the text data segment.

[0100] If a text data segment belongs to a sentence containing a valid noun in a predetermined valid noun set, the present disclosure may directly regard the text data segment as a valid text segment, and the text data segment may be regarded as a non-standard valid clause of the text data segment.

[0101] If a text data segment neither belongs to a sentence containing a predetermined word based on affiliation and a valid noun in a predetermined valid noun set, nor does it belong to a sentence containing a valid noun in a predetermined valid noun set, the present disclosure may mark the text data segment as an invalid clause or as a text data segment that does not require text extraction processing. The present disclosure may not perform text extraction processing operations on the text data segment, that is, the valid text segment corresponding to the text data segment is empty.

[0102] In an optional example, the effective noun set of the present disclosure may be set using a plurality of text data to be processed. An example of setting the effective noun set of the present disclosure is as follows: Figure 3 shown.

[0103] Figure 3 In step S300, part-of-speech segmentation is performed on the plurality of text data to be processed belonging to the second data item to obtain the part-of-speech of each word contained in each of the plurality of text data to be processed.

[0104] Optionally, the present disclosure may use tools such as Jieba word segmentation to perform part-of-speech segmentation on multiple text data to be processed. Parts of speech in the present disclosure may refer to nouns, verbs, auxiliary words, and prepositions.

[0105] S301, performing word frequency statistics on all words whose part of speech is noun, and obtaining word frequency statistics results of each word whose part of speech is noun.

[0106] S302: Select nouns whose frequency statistics satisfy a predetermined frequency condition from all words whose part of speech is noun, to form a valid noun set.

[0107] Optionally, the present disclosure may add all nouns with a frequency greater than or equal to a predetermined frequency in the word frequency statistics results to the effective noun set.

[0108] The present disclosure obtains a valid noun set by utilizing a plurality of to-be-processed text data belonging to the second data item, which is conducive to conveniently obtaining a targeted valid noun set, thereby facilitating accurate acquisition of valid text data of the to-be-processed text data.

[0109] In an optional example, the valid text data obtained by the present disclosure can be used to generate a multimedia file for the object to be processed. For example, in the process of generating a multimedia file for the object to be processed, the data item to which the text data to be processed in the present disclosure belongs can be used as an information display dimension of the object to be processed (such as the first data item is an information display dimension of the object to be processed, and the second data item is another information display dimension of the object to be processed), and the valid text data of the text data to be processed can be the content of the information display dimension. The present disclosure can generate a multimedia unit in the multimedia file of the object to be processed according to the content of the information display dimension of the object to be processed. For example, when the content of the information display dimension of the object to be processed meets the requirements of the multimedia unit (such as the number of words meets the requirements of the multimedia unit, etc.), the present disclosure can display the content of the information display dimension of the object to be processed in the form of text in a multimedia unit of the multimedia file of the object to be processed. In addition, in the process of displaying the content of the information display dimension of the object to be processed in the form of text in the multimedia unit, the present disclosure can set the font and font size of the content of the information display dimension according to the requirements of the multimedia unit on the number of words, so that the content of the information display dimension fills the corresponding text display area in the multimedia unit as much as possible in a predetermined manner. In a more specific example, a multimedia unit in the multimedia file of the object to be processed is presented as a table. The present disclosure can insert the content of the information display dimension of the object to be processed into a cell of the table, and the present disclosure can adjust the font size of the content of the information display dimension according to the requirement on the number of words of the cell in the table.

[0110] In one example, the present disclosure can perform dimension commonality detection processing on the contents of all information display dimensions of multiple objects to be processed, and use the result of the dimension commonality detection processing to determine the contents of each information display dimension of each object to be processed, thereby using the determined contents of each information display dimension of each object to be processed to form a multimedia file of each object to be processed. Figure 4 shown.

[0111] S400: Obtain all information display dimensions of multiple objects to be processed.

[0112] The object to be processed in the present disclosure may refer to an object that needs to be explained or introduced. The specific form of expression of the object to be processed is usually related to the actual field to which the method of the present disclosure is applicable. For example, when the method of the present disclosure is applicable to the field of company information query, the object to be processed may be a company that needs to be explained or introduced. For another example, when the method of the present disclosure is applicable to the field of product sales, the object to be processed may be a product that needs to be explained or introduced. The present disclosure does not limit the specific form of expression of the object to be processed.

[0113] The information display dimensions in the present disclosure can be considered as angles used to describe the object to be processed, or can be considered as information items used to describe the object to be processed. Under normal circumstances, it is often necessary to describe an object to be processed from multiple different information display dimensions, that is, an object to be processed usually contains multiple information display dimensions. For example, assuming that the object to be processed is a company, the present disclosure can describe the object to be processed from multiple information display dimensions such as "company name", "company legal person", "company registration address", "company registration time", "company staff size" and "company business scope".

[0114] It should be noted that all information display dimensions contained in one object to be processed may not be completely the same as all information display dimensions contained in another object to be processed, and all information display dimensions contained in one object to be processed may be completely the same as all information display dimensions contained in another object to be processed. All information display dimensions contained in each object to be processed may be determined by the properties of the object to be processed (such as whether it is a subsidiary, whether it belongs to a special industry company), etc.

[0115] The present disclosure can obtain all information display dimensions of multiple objects to be processed by summarizing all information display dimensions contained in each of the multiple objects to be processed. All information display dimensions of multiple objects to be processed can form an information display dimension set. In other words, the information display dimensions of all objects to be processed should belong to the information display dimension set. The present disclosure can determine the information display dimensions that should be included in the information display dimension set based on factors such as the respective attributes of all objects to be processed. The present disclosure does not limit the specific expression form of the information display dimensions contained in the information display dimension set.

[0116] S401. For at least one information display dimension among all the above-mentioned information display dimensions, according to the content of the at least one information display dimension of multiple objects to be processed, perform dimension commonality detection on the at least one information display dimension to determine whether the at least one information display dimension is an information display dimension with dimension commonality.

[0117] The present disclosure can perform dimension commonality detection processing on all information display dimensions separately, and can also perform dimension commonality detection processing on some information display dimensions (such as one or two information display dimensions) among all information display dimensions. For any information display dimension, whether the information display dimension has dimension commonality is usually determined by the content of the information display dimension of all objects to be processed.

[0118] The content of the information display dimension in the present disclosure can be considered as the value of the information display dimension. The value here can include at least one of text form and image form. The text can include words and numbers, etc. For any information display dimension, the content of the information display dimension of two different objects to be processed can be completely the same, completely different, or partially the same.

[0119] The dimension commonality detection disclosed in the present invention may refer to a detection for determining whether the common points of the values ​​of multiple objects to be processed on an information display dimension are compliant. For an information display dimension, the present invention may set conditions for the information display dimension in advance, so that it can be determined whether the common points of the content meet the conditions based on the content of the information display dimension of multiple objects to be processed, so as to determine whether the information display dimension is an information display dimension with dimensional commonality. For example, for an information display dimension, if the common points of the content of the information display dimension of multiple objects to be processed meet the conditions, then the information display dimension is determined to be an information display dimension with dimensional commonality, otherwise it is determined that the information display dimension is not an information display dimension with dimensional commonality.

[0120] S402: For any information display dimension with dimensional commonality, update the content of the information display dimension using the commonality label of the information display dimension.

[0121] Any information display dimension with dimensional commonality in the present disclosure has at least one common label. The common label in the present disclosure may refer to a label that can summarize multiple contents (such as all contents or partial contents) in all contents of the same information display dimension of two or more objects to be processed. Any information display dimension with dimensional commonality may have multiple common labels. For example, assuming that the first information display dimension is an information display dimension among all information display dimensions of N (N is a positive integer greater than 2) objects to be processed, assuming that the first information display dimension has two common labels, one of which is: a first label that can summarize all contents of the first information display dimension of the first n (n is a positive integer less than N) objects to be processed among the N objects to be processed, and the other common label is: a second label that can summarize all contents of the first information display dimension of the last Nn objects to be processed among the N objects to be processed.

[0122] For any information display dimension with dimensional commonality, the present disclosure can use the common label of the information display dimension to replace the content of the information display dimension in multiple objects to be processed. Continuing with the previous example, the content of the first information display dimension of the first n objects to be processed is replaced with the first label, and the content of the first information display dimension of the next Nn objects to be processed is replaced with the second label.

[0123] The present disclosure may use the FastText model, etc. to set commonality tags for the content of the information display dimension with dimensional commonality. For example, the FastText model may be used to set four commonality tags, "0-10 people", "10-100 people", "100-500 people" and "greater than 500 people", for the content of the information display dimension of the company's personnel scale with dimensional commonality. For another example, the commonality tags set for the content of the information display dimension of "company business scope" with dimensional commonality using the FastText model may include but are not limited to: technical services, water pollution control, cargo import and export, loading and unloading, clothing wholesale, parking lot operation, sales of software products, commodity retail trade, technology import and export, education consulting and commission agency, etc. In addition, the present disclosure may also use the preset commonality tags as the commonality tags for the content of the information display dimension with dimensional commonality. For example, a commonality tag set may be pre-set. When an information display dimension has dimensional commonality, the corresponding commonality tags may be selected from the commonality tag set according to the content of the information display dimension, and the selected commonality tags may be used as the commonality tags for the content of the information display dimension with dimensional commonality. The present disclosure does not limit the specific implementation method of setting the common tags.

[0124] S403: Generate multiple multimedia units using all information display dimensions and their contents.

[0125] In the case where all information display dimensions in this application are information display dimensions with dimensional commonality, all information display dimensions in this application and their contents include: all information display dimensions with dimensional commonality and their updated contents. In the case where all information display dimensions in this application include: information display dimensions with dimensional commonality and information display dimensions without dimensional commonality, all information display dimensions in this application and their contents include: information display dimensions with dimensional commonality and their updated contents, and information display dimensions without dimensional commonality and their initial contents.

[0126] For any information display dimension, the information display dimension corresponds to different multimedia units according to its different common labels, that is, the information display dimension corresponds to different multimedia units when it has different common labels. Continuing with the previous example, the first information display dimension and the first label correspond to one multimedia unit, and the first information display dimension and the second label correspond to another multimedia unit. The multimedia unit in the present disclosure can be played, and when the multimedia unit is played, the corresponding picture and sound are usually presented. The picture is usually formed based on the pre-set surface background, the information display dimension and its content. The sound is usually formed based on the information display dimension and its content, and can also be formed based on the background sound, the information display dimension and its content.

[0127] The multimedia unit in the present disclosure may also be referred to as a rich media unit. The multimedia unit in the present disclosure may be considered as a component unit of a multimedia file, and a multimedia unit may be considered as a video segment or a video unit, that is, a multimedia unit may include multiple video frames.

[0128] In the present disclosure, one information display dimension usually corresponds to multiple multimedia units. The number of multimedia units corresponding to one information display dimension is usually related to the number of contents of the information display dimension (the content of the information display dimension updated with the common tag). For example, assuming that the number of contents of an information display dimension is a (a is an integer greater than 0), then the information display dimension can correspond to a multimedia units.

[0129] All multimedia units generated by the present disclosure can form a multimedia unit set. The multimedia files of each object to be processed are formed by the multimedia units in the multimedia unit set. The present disclosure can use a variety of existing tools for generating videos to form multimedia units. The present disclosure does not limit the specific process of generating multimedia units.

[0130] S404: For any object to be processed, select multimedia units belonging to the object to be processed from multimedia units according to all information display dimensions and contents contained in the object to be processed, and synthesize all selected multimedia units into a multimedia file of the object to be processed.

[0131] After selecting all multimedia units belonging to an object to be processed from a multimedia unit set, the present invention can sort all the selected multimedia units according to a predetermined order (i.e., a pre-set arrangement order of information display dimensions, for example, the company legal person and company executives information should be set after the company name, etc.), and use tools such as video splicing to perform splicing processing on all the sequentially arranged multimedia units, thereby forming a multimedia file of the object to be processed.

[0132] The present disclosure can obtain information display dimensions with dimensional commonality by performing dimensional commonality detection on at least one information display dimension respectively; for information display dimensions with dimensional commonality, the present disclosure updates the content of the information display dimension by using the commonality tags of the information display dimension, so that when all information display dimensions and their contents are used to generate multiple multimedia units, it is not only conducive to reducing the number of generated multimedia units to a large extent, but also conducive to avoiding the phenomenon that the same information is expressed in a variety of different ways; by selecting each multimedia unit belonging to an object to be processed from the generated multimedia units, and using all the selected multimedia units to synthesize the multimedia file of the object to be processed, it is not only conducive to conveniently forming the multimedia files of all objects to be processed, but also conducive to avoiding the phenomenon that different multimedia files use different multimedia units to express the same information, which brings confusion to the viewers of the multimedia files. It can be seen that the embodiments of the present disclosure are conducive to improving the generation efficiency of multimedia files and improving the standardization of multimedia files.

[0133] In an optional example, the commonality extraction processing in the present disclosure may refer to a process for dividing all contents of the information display dimension of all objects to be processed into one or more groups for any information display dimension, wherein all contents in any group have common characteristics.

[0134] Optionally, for any information display dimension, there may be a phenomenon that all contents of the information display dimension have multiple common features. For example, the first x1 (x1 is an integer greater than 1) contents of the information display dimension have a first common feature, the middle x2 (x2 is an integer greater than 1) contents of the information display dimension have a second common feature, and the last x3 (x3 is an integer greater than 1) contents of the information display dimension have a third common feature. Of course, there may also be a possibility that a content of the information display dimension does not have common features with any other content of the information display dimension.

[0135] Optionally, the present disclosure may pre-set corresponding conditions for commonality extraction. If the result of the commonality extraction processing for an information display dimension meets the pre-set corresponding conditions, it can be determined that the information display dimension is an information display dimension with dimensional commonality. If the result of the commonality extraction processing for an information display dimension does not meet the pre-set corresponding conditions, it can be determined that the information display dimension is not an information display dimension with dimensional commonality.

[0136] In an optional example, for any information display dimension, the present disclosure may adopt a clustering approach to achieve commonality extraction processing of the content of the information display dimension after preprocessing of all objects to be processed. Specifically, for any information display dimension, the present disclosure may perform clustering processing on the content of the information display dimension of all objects to be processed, thereby obtaining multiple clusters; all the content of the information display dimension contained in each cluster can be considered to have commonality. The present disclosure may use a variety of clustering algorithms for clustering processing, and the present disclosure does not limit this.

[0137] Optionally, in the case where the commonality extraction processing is a clustering processing, the pre-set corresponding conditions in the present disclosure may be predetermined conditions set for the number of clusters, that is, if the number of all clusters obtained by the clustering processing for an information display dimension meets the predetermined conditions (such as being less than or equal to a predetermined number, etc.), then the information display dimension is determined to be an information display dimension with dimensional commonality; if the number of all clusters obtained by the clustering processing for an information display dimension does not meet the predetermined conditions (such as being greater than a predetermined number, etc.), then the information display dimension is determined to be an information display dimension without dimensional commonality.

[0138] The present disclosure clusters the contents of an information display dimension of all objects to be processed, which is conducive to objectively measuring information display dimensions with dimensional commonality and information display dimensions without dimensional commonality, thereby facilitating improving the accuracy of determining information display dimensions with dimensional commonality.

[0139] In an optional example, the present disclosure utilizes all information display dimensions and their contents to generate an example of multiple multimedia units such as Figure 5 shown.

[0140] Figure 5 In S500, for any information display dimension, multiple audio data are generated using the information display dimension and content.

[0141] Optionally, the present disclosure may use an information display dimension and its content as input parameters of an interface for generating audio, thereby obtaining multiple audio data according to the return of the interface. One audio data may be considered as an audio file.

[0142] For an information display dimension, the number of audio files generated for the information display dimension and its content is usually related to the number of contents of the information display dimension. For example, if an information display dimension has i different contents, the present disclosure generates an audio file for the information display dimension and each content, thereby obtaining i audio files. In a more specific example, if an information display dimension has j common tags, the present disclosure can generate an audio file for the information display dimension and each common tag, thereby obtaining j audio files.

[0143] It should be noted that the present disclosure can generate multiple audio data according to all information display dimensions and their contents before the dimensional commonality detection, and generate multiple audio data according to the information display dimensions with dimensional commonality and their commonality tags after the dimensional commonality detection. The present disclosure can also generate multiple audio data according to all information display dimensions and their contents after updating the content of the corresponding information display dimensions using the commonality tags. This method is conducive to avoiding the execution of some audio data generation processes.

[0144] S501 . For any information display dimension, according to the information display dimension and its content, using a picture frame template corresponding to the information display dimension, a picture frame of the information display dimension is generated.

[0145] Optionally, the picture frame template in the present disclosure can be considered as a background template. One information display dimension can correspond to multiple picture frame templates, and different picture frame templates can correspond to different contents of the type information display dimension. In other words, the present disclosure can classify the contents of the information display dimension, and a type of content in the information display dimension corresponds to a picture frame template. Different categories of content in the information display dimension correspond to different picture frame templates. The present disclosure can classify the contents of the information display dimension according to actual needs.

[0146] Optionally, for any content of any information display dimension, the present disclosure may first determine the category to which the content of the information display dimension belongs, and determine the screen frame template corresponding to the information display dimension and its content according to the category; then, the screen frame template, the information display dimension and its content are used to generate a screen frame of the information display dimension. That is, the present disclosure fills the information display dimension and its content into the screen frame template based on the predetermined format of the corresponding screen frame template, thereby forming a screen frame of the information display dimension.

[0147] Optionally, the picture frame of the information display dimension in the present disclosure may be a video frame or a bitmap image, etc. The present disclosure may pre-set at least one picture frame template for each information display dimension. The picture frame template usually pre-sets the corresponding picture style and information / picture element filling position for the information display dimension and its content. The picture elements here may include business cards and decorative patterns, etc. In one example, the above-mentioned business card may be a legal person's business card or an ultimate beneficiary's business card, etc.

[0148] Optionally, the present disclosure may first discretize the picture frame template of the information display dimension to obtain multiple picture frame templates, and based on the predetermined format of the corresponding picture frame templates, fill the information display dimension and its content into the multiple picture frame templates to form a picture frame of the information display dimension.

[0149] S502: For any information display dimension, merge the audio data of the information display dimension and the picture frame of the information display dimension into a multimedia unit of the information display dimension.

[0150] Optionally, the present disclosure may utilize a tool for matching audio tracks and pictures to merge audio data of an information display dimension and all picture frames of the information display dimension into a multimedia unit with synchronized sound and picture.

[0151] The present disclosure utilizes information display dimensions and content (including updated content), which can not only conveniently generate corresponding audio data, but also helps to reduce the amount of generated audio data, thereby helping to improve the efficiency of generating audio data, and further helping to improve the generation efficiency of multimedia units.

[0152] In an optional example, the present disclosure generates an example of a multimedia unit of an information display dimension such as Figure 6 shown.

[0153] Figure 6 In S600, for any information display dimension, a subtitle corresponding to the information display dimension is formed according to the information display dimension and its content.

[0154] Optionally, the present disclosure may adopt a method of combining the information display dimension and its content to obtain subtitles of the information display dimension. In the process of combining the information display dimension and its content, attributives, conjunctions, modal particles, and structural particles may be added between the information display dimension and its content or after the content, so that the subtitles are more in line with the browsing habits of multimedia file viewers. For example, assuming that an information display dimension is "company business scope", and the content of the information display dimension is "investment management" and "asset management", the subtitles corresponding to the information display dimension may be "company business scope includes: investment management and asset management, etc."

[0155] S601, setting the subtitles at a predetermined position in the picture frame of the information display dimension.

[0156] Optionally, the present disclosure pre-sets the position of the subtitles formed by the information display dimension and its content in the picture frame, and the present disclosure can have corresponding settings for the font, font size and font color of the subtitles. The present disclosure can set the subtitles in the picture frame based on the pre-setting.

[0157] S602: Adjust the number of picture frames in the information display dimension according to the playback time length of the audio data in the information display dimension and a preset frame rate.

[0158] Optionally, the purpose of adjusting the number of picture frames of the information display dimension in the present disclosure includes: making the playback time length of the audio data of the information display dimension substantially the same as the playback time length of the picture frames of the information display dimension. The preset frame rate in the present disclosure may refer to the frequency of picture frames appearing continuously on the display screen per unit time. The preset frame rate may specifically be the number of picture frames appearing continuously on the display screen per second. The size of the preset frame rate in the present disclosure is usually set according to the actual situation of the application that plays the multimedia file (such as an application such as a video playback tool).

[0159] Optionally, the present disclosure can adjust the number of picture frames of the information display dimension by increasing or decreasing the picture frames. For example, assuming that the playback time length of the audio data of an information display dimension is a1 seconds, and the preset frame rate is a2 frames / second, then the number of picture frames required for the information display dimension is a3, which is the product of a1 and a2. If the number of picture frames corresponding to the information display dimension is less than a3, the number of picture frames corresponding to the information display dimension should be increased to a3; if the number of picture frames corresponding to the information display dimension is greater than a3, the number of picture frames corresponding to the information display dimension should be reduced to a3.

[0160] S603: Merge the audio data of the information display dimension and the picture frames after the number of the information display dimension is adjusted into a multimedia unit of the information display dimension.

[0161] Optionally, the present disclosure may utilize a tool for matching audio tracks and pictures to merge audio data of an information display dimension and all picture frames of the information display dimension into a multimedia unit with synchronized sound and picture.

[0162] The present disclosure sets subtitles for picture frames and adjusts the number of picture frames based on the playback time length of audio data and a preset frame rate, which helps to synchronize the subtitles and sounds in the multimedia unit of the information display dimension with the picture, thereby helping to improve the quality of the multimedia unit.

[0163] In an optional example, the present disclosure combines all selected multimedia units into an example of a multimedia file to be processed. Figure 7 shown.

[0164] Figure 7 In S700, for any object to be processed, all multimedia units selected based on the object to be processed are spliced ​​according to a predetermined arrangement order of the multimedia units to obtain an intermediate multimedia file of the object to be processed.

[0165] Optionally, the predetermined arrangement order of the multimedia units in the present disclosure may be set according to the actual display requirements of the information display dimension. For example, when the method of the present disclosure is applicable to the field of company information query, and the object to be processed is a company that needs to be explained or introduced, the predetermined arrangement order of the multimedia units may be: first, "company name", then "company legal person", then "company nature", then "company registration time", then "company registration address", then "company business scope"... and so on. The present disclosure does not limit the predetermined arrangement order of the multimedia units.

[0166] Optionally, after the present disclosure sequentially arranges all the selected multimedia units belonging to the same object to be processed according to the predetermined arrangement order of the multimedia units, it can use tools such as video splicing to splice all the sequentially arranged multimedia units, so that the splicing processing result of the video splicing tool can be used as the intermediate multimedia file of the object to be processed. The intermediate multimedia file of the present disclosure can be considered as the multimedia file to be processed, that is, the intermediate multimedia file is a multimedia file, and it is a multimedia file that needs to be further processed.

[0167] S701: Post-process the intermediate multimedia file of the object to be processed to obtain the multimedia file of the object to be processed.

[0168] Optionally, the post-processing in the present disclosure may include at least one of: a playback rate adjustment process and a file format conversion process. The playback rate adjustment process may refer to a process for adjusting the playback speed of the intermediate multimedia file, for example, the playback rate adjustment process may be a double speed adjustment process or a triple speed adjustment process or other M (M is a decimal or integer) speed adjustment process. The file format conversion process may refer to a process for changing the file format of the intermediate multimedia file. For example, the file format conversion process may be a file compression process, etc.

[0169] As a more specific example, assuming that the file format of the intermediate multimedia file of the present disclosure is MPEG (Moving Picture Experts Group), the present disclosure first performs M-fold speed adjustment processing on the intermediate multimedia file, and the audio playback effect of the multimedia file after the speed adjustment processing should be close to the normal speaking speed of human beings; thereafter, the present disclosure can perform file format conversion processing on the multimedia file after the speed adjustment processing, so that the file format of the multimedia file after the speed adjustment processing is converted from MPEG format to compression format such as rar, thereby helping to reduce the storage space required for the multimedia file of the object to be processed. The post-processing of the present disclosure can be set according to actual needs, and the present disclosure does not limit the specific form of post-processing. The multimedia file of the object to be processed finally formed by the present disclosure can be a video of the object to be processed in seconds.

[0170] The present disclosure performs splicing and post-processing on all selected multimedia units of an object to be processed based on a predetermined arrangement order of the multimedia units, which is beneficial to optimizing the multimedia file of the object to be processed, thereby providing a better user experience for viewers of the multimedia files.

[0171] Figure 8 The figure is a schematic diagram of the structure of an embodiment of a text data processing device of the present disclosure. The text data processing device of this embodiment can be used to implement the above-mentioned embodiments of the text data processing method of the present disclosure.

[0172] Figure 8 The device shown includes: a text data acquisition module 800, a segmentation processing module 801, a classification processing module 802, an extraction processing module 803, and a valid text data module 804. Optionally, the device of the present disclosure may also include: an effective noun set acquisition module 805, a display dimension acquisition module 806, a dimension commonality determination module 807, a dimension content update module 808, a media unit generation module 809, and a media file synthesis module 810.

[0173] The text data acquisition module 800 is used to acquire text data to be processed.

[0174] Optionally, the module for acquiring text data 800 may first acquire long text data and determine a dirty word set corresponding to the data item to which the long text data belongs. Afterwards, the module for acquiring text data 800 performs data cleaning on the long text data according to the dirty word set corresponding to the data item to which the long text data belongs, and uses the result of the data cleaning as the text data to be processed. The long text data includes: text data whose byte length meets the predetermined byte length requirement. In addition, the dirty word sets corresponding to at least two data items are different.

[0175] The segmentation processing module 801 is used to segment the text data to be processed obtained by the text data obtaining module 800 to obtain multiple text data segments.

[0176] The classification processing module 802 is used to classify the multiple text data segments obtained by the segmentation processing module 801 according to the classification method corresponding to the data item to which the text data to be processed belongs, and obtain the classification processing results of each of the multiple text data segments, wherein the classification methods corresponding to at least two data items are different.

[0177] Optionally, for any text data segment, the classification processing module 802 can determine the classification condition that the text data segment satisfies for the first time according to the priority of multiple classification conditions contained in the classification method corresponding to the data item to which the text data to be processed belongs, and use the classification condition that is satisfied for the first time as the classification processing result of the text data segment.

[0178] Optionally, when the data item to which the text data to be processed belongs is the first data item, the multiple classification conditions contained in the classification method corresponding to the first data item include at least one of the following, and the priority of the multiple classification conditions is arranged in the following order from high to low: whether the text data segment belongs to a legal sentence containing predetermined legal terms; whether the text data segment belongs to an introductory sentence containing a colon; whether the text data segment belongs to a gerund sentence containing a colon; whether the text data segment belongs to a chemical formula sentence containing predetermined characters based on a chemical formula; whether the text data segment belongs to a sentence containing predetermined auxiliary words; whether the text data segment belongs to a sentence containing predetermined punctuation marks.

[0179] Optionally, when the data item to which the text data to be processed belongs is the second data item, the multiple classification conditions contained in the classification method corresponding to the second data item include at least one of the following, and the priority of the multiple classification conditions is arranged in the following order from high to low: whether the text data segment belongs to a sentence containing predetermined words based on affiliation and valid nouns in a predetermined valid noun set; whether the text data segment belongs to a sentence containing valid nouns in a predetermined valid noun set.

[0180] The extraction processing module 803 is used to perform text extraction processing on the multiple text data segments respectively according to the text extraction methods corresponding to the classification processing results of the multiple text data segments, so as to obtain valid text segments corresponding to the multiple text data segments respectively.

[0181] Optionally, the extraction processing module 803 may not perform text extraction processing on the legal sentence, and the valid text segment of the legal sentence is empty.

[0182] Optionally, the extraction processing module 803 may extract the sentence after the colon from the introductory sentence, and use the sentence after the colon as the valid text segment of the introductory sentence.

[0183] Optionally, the extraction processing module 803 may generate a valid text segment of a noun plus a predetermined particle plus a verb according to the verb and the noun in the gerund sentence.

[0184] Optionally, the extraction processing module 803 may take a predetermined number of chemical formulas in the chemical formula statement as valid text segments.

[0185] Optionally, the extraction processing module 803 may generate a valid text segment of noun plus predetermined particle plus verb according to the verb and noun in the sentence containing the predetermined particle.

[0186] Optionally, the extraction processing module 803 may segment the text data segment according to predetermined punctuation marks contained in the text data segment, and use each segment obtained by the segmentation as a valid text segment.

[0187] Optionally, the extraction processing module 803 may take a text data segment containing a predetermined term based on affiliation and a valid noun in a predetermined valid noun set as a valid text segment.

[0188] Optionally, the extraction processing module 803 may take a text data segment containing a valid noun in a predetermined valid noun set as a valid text segment.

[0189] The valid text data module 804 is used to use the valid text segments corresponding to each of the multiple text data segments as valid text data of the text data to be processed.

[0190] The module 805 for obtaining a valid noun set is used to perform part-of-speech segmentation processing on multiple text data to be processed belonging to the second data item, obtain the part-of-speech of each word contained in each of the multiple text data to be processed, and perform word frequency statistics on all words whose part of speech is noun, obtain the word frequency statistics results of each word whose part of speech is noun, and select nouns whose word frequency statistics results meet predetermined word frequency conditions from all words whose part of speech is noun, to form a valid noun set.

[0191] The display dimension acquisition module 806 is used to acquire all information display dimensions of multiple objects to be processed.

[0192] The dimension commonality determination module 807 is used to perform dimension commonality detection on at least one information display dimension among all the information display dimensions obtained by the display dimension acquisition module 806, based on the content of at least one information display dimension of multiple objects to be processed, to determine whether at least one information display dimension is an information display dimension with dimension commonality.

[0193] The update dimension content module 808 is used to update the content of any information display dimension with dimensional commonality determined by the determination dimension commonality module 807 using the commonality label of the information display dimension, wherein any information display dimension with dimensional commonality has at least one commonality label.

[0194] The media unit generation module 809 is used to generate multiple multimedia units using all the information display dimensions and their contents that have been updated and processed by the dimension content update module 808, wherein one information display dimension corresponds to different multimedia units according to its different common labels, and all the information display dimensions and their contents include: information display dimensions with dimensional commonality and their updated contents.

[0195] The synthetic media file module 810 is used to select multimedia units belonging to any object to be processed from all multimedia units generated by the media unit generation module 809 according to all information display dimensions and contents contained in the object to be processed, and synthesize all selected multimedia units into a multimedia file of the object to be processed.

[0196] In an optional example, for any information display dimension, the module for determining dimension commonality 807 may first cluster the effective contents of the information display dimension of multiple objects to be processed, so as to obtain multiple clusters; thereafter, the module for determining dimension commonality 807 may judge whether the number of multiple clusters meets a predetermined condition; if the number of multiple clusters meets the predetermined condition, the module for determining dimension commonality 807 may determine that the information display dimension is an information display dimension with dimension commonality; otherwise, the module for determining dimension commonality 807 may determine that the information display dimension is an information display dimension without dimension commonality.

[0197] In an optional example, the media unit generation module 809 disclosed in the present invention may include: an audio generation submodule 8091, a picture frame generation submodule 8092, and a merging submodule 8093. The audio generation submodule 8091 is used to generate multiple audio data for any information display dimension using the information display dimension and content, wherein one information display dimension corresponds to different audio data according to its different common labels. The picture frame generation submodule 8092 is used to generate a picture frame of any information display dimension according to the information display dimension and its content using a picture frame template corresponding to the information display dimension. The merging submodule 8093 is used to merge the audio data of any information display dimension with the picture frame of the information display dimension into a multimedia unit of the information display dimension.

[0198] In an optional example, the merging submodule 8093 can form subtitles corresponding to the information display dimension based on the information display dimension and its content, and set the subtitles at a predetermined position in the picture frame of the information display dimension. Afterwards, the merging submodule 8093 can adjust the number of picture frames of the information display dimension according to the playback time length of the audio data of the information display dimension and the preset frame rate, and merge the audio data of the information display dimension and the picture frames after the number of the information display dimension is adjusted into a multimedia unit of the information display dimension.

[0199] In an optional example, for any object to be processed, the synthesized media file module 810 may perform splicing processing on all the selected multimedia units according to the predetermined arrangement order of the multimedia units to obtain the intermediate multimedia file of the object to be processed; thereafter, the synthesized media file module 810 performs post-processing on the intermediate multimedia file of the object to be processed to obtain the multimedia file of the object to be processed. The post-processing may include at least one of: a playback rate adjustment process and a file format conversion process.

[0200] In addition, an embodiment of the present disclosure further provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the text data processing method described in any of the above embodiments of the present disclosure is implemented.

[0201] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the text data processing method described in any of the above embodiments of the present disclosure is implemented.

[0202] Fig. 9 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Fig. 9 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0203] like Fig. 9 As shown, the electronic device includes one or more processors and memory.

[0204] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0205] The memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the text data processing method of each embodiment of the present disclosure described above and / or other desired functions.

[0206] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown). In addition, the input device may also include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and a remote output device connected thereto, etc. Of course, for simplicity, Figure 8 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to the application.

[0207] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to execute the steps in the text data processing method according to various embodiments of the present disclosure described in the above part of this specification. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for executing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0208] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the text data processing method according to various embodiments of the present disclosure described in the above part of this specification.

[0209] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0210] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0211] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As those skilled in the art will recognize, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0212] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0213] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0214] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0215] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A text data processing method, characterized in that: include: Get the text data to be processed; Segmenting the text data to be processed to obtain multiple text data segments; According to the classification methods corresponding to the data items to which the text data to be processed belongs, the plurality of text data segments are classified and processed respectively to obtain classification processing results of the plurality of text data segments, wherein the classification methods corresponding to at least two data items are different; According to the text extraction methods corresponding to the classification processing results of the multiple text data segments, respectively, the multiple text data segments are subjected to text extraction processing to obtain valid text segments corresponding to the multiple text data segments; Using the valid text segments corresponding to each of the plurality of text data segments as valid text data of the text data to be processed; The method of classifying the plurality of text data segments according to the classification method corresponding to the data item to which the to-be-processed text data belongs, and obtaining classification processing results of each of the plurality of text data segments, comprises: for any text data segment, determining the classification condition that the text data segment satisfies for the first time according to the priorities of the plurality of classification conditions included in the classification method corresponding to the data item to which the to-be-processed text data belongs, and taking the classification condition that is satisfied for the first time as the classification processing result of the text data segment; Generating a multimedia unit in a multimedia file of the object to be processed according to the content of the information display dimension of the object to be processed, comprising: acquiring all information display dimensions of a plurality of objects to be processed; for at least one information display dimension among all the information display dimensions, performing dimension commonality detection on the at least one information display dimension according to the content of the at least one information display dimension of the plurality of objects to be processed, so as to determine whether the at least one information display dimension is respectively an information display dimension with dimension commonality; for any information display dimension with dimension commonality, updating the content of the information display dimension using the commonality label of the information display dimension, wherein any information display dimension with dimension commonality has at least one commonality label; generating a plurality of multimedia units using all the information display dimensions and their contents, wherein one information display dimension corresponds to different multimedia units according to its different commonality labels, and all the information display dimensions and their contents include: information display dimensions with dimension commonality and their updated contents; for any object to be processed, selecting a multimedia unit belonging to the object to be processed from the multimedia units according to all the information display dimensions and their contents contained in the object to be processed, and synthesizing all the selected multimedia units into a multimedia file of the object to be processed.

2. The method according to claim 1, characterized in that The step of obtaining the text data to be processed includes: Acquire long text data, wherein the long text data includes: text data whose byte length meets a predetermined byte length requirement; Determine a dirty word set corresponding to the data item to which the long text data belongs, wherein the dirty word sets corresponding to at least two data items are different; According to the dirty word set corresponding to the data item to which the long text data belongs, the long text data is cleaned, and the result of the data cleaning is used as the text data to be processed.

3. The method according to claim 1, characterized in that In the case where the data item to which the to-be-processed text data belongs is the first data item, the multiple classification conditions included in the classification method corresponding to the first data item include at least one of the following, and the priorities of the multiple classification conditions are arranged in the following order from high to low: Whether the text data segment belongs to a legal sentence containing predetermined legal terms; Whether the text data segment is an introductory statement containing a colon; Whether the text data segment belongs to a gerund sentence containing a colon; whether the text data segment belongs to a chemical formula sentence containing predetermined characters based on the chemical formula; Whether the text data segment belongs to a sentence containing a predetermined particle; Whether the text data segment belongs to a sentence containing predetermined punctuation marks.

4. The method according to claim 3, characterized in that The step of performing text extraction processing on the multiple text data segments respectively according to the text extraction methods corresponding to the classification processing results of the multiple text data segments to obtain valid text segments corresponding to the multiple text data segments respectively includes at least one of the following: No text extraction process is performed on the legal sentence, and the valid text segment of the legal sentence is empty; Extracting the sentence after the colon from the introductory sentence, and taking the sentence after the colon as the valid text segment of the introductory sentence; According to the verb and the noun in the gerund sentence, a valid text segment of the noun plus the predetermined particle plus the verb is generated; taking a predetermined number of chemical formulas in the chemical formula sentence as valid text segments; According to the verbs and nouns in the sentence containing the predetermined particle, generating a valid text segment of the noun plus the predetermined particle plus the verb; The text data segment is segmented according to predetermined punctuation marks contained in the text data segment, and each segment obtained by the segmentation process is used as a valid text segment.

5. The method according to claim 1, characterized in that In the case where the data item to which the text data to be processed belongs is the second data item, the multiple classification conditions included in the classification method corresponding to the second data item include at least one of the following, and the priorities of the multiple classification conditions are arranged in the following order from high to low: whether the text data segment belongs to a sentence including a predetermined term based on affiliation and a valid noun in a predetermined valid noun set; Whether the text data segment belongs to a sentence containing a valid noun in a predetermined valid noun set.

6. The method according to claim 5, characterized in that The step of performing text extraction processing on the multiple text data segments respectively according to the text extraction methods corresponding to the classification processing results of the multiple text data segments to obtain valid text segments corresponding to the multiple text data segments respectively includes at least one of the following: Taking a text data segment including a predetermined word based on affiliation and a valid noun in a predetermined valid noun set as a valid text segment; A text data segment including a valid noun in a predetermined valid noun set is regarded as a valid text segment.

7. The method according to claim 5 or 6, characterized in that: The method further comprises: Performing part-of-speech segmentation processing on the plurality of text data to be processed belonging to the second data item to obtain the part-of-speech of each word contained in each of the plurality of text data to be processed; Perform word frequency statistics on all words whose part of speech is noun, and obtain word frequency statistics results of each word whose part of speech is noun; Nouns whose word frequency statistics satisfy a predetermined word frequency condition are selected from all words whose part of speech is noun to form the valid noun set.

8. The method according to any one of claims 1 to 6, characterized in that The data item to which the text data to be processed belongs is used as the information display dimension of the object to be processed, and the valid text data of the text data to be processed is used as the content of the information display dimension of the object to be processed.

9. A text data processing device, characterized in that: include: A text data acquisition module is used to acquire text data to be processed; A segmentation processing module, used for segmenting the text data to be processed to obtain multiple text data segments; A classification processing module, used for classifying the plurality of text data segments according to the classification methods corresponding to the data items to which the text data to be processed belongs, to obtain classification processing results of the plurality of text data segments, wherein the classification methods corresponding to at least two data items are different; An extraction processing module, used to perform text extraction processing on the multiple text data segments respectively according to the text extraction methods corresponding to the classification processing results of the multiple text data segments, so as to obtain valid text segments corresponding to the multiple text data segments respectively; A valid text data module, used for taking the valid text segments corresponding to each of the plurality of text data segments as the valid text data of the text data to be processed; The classification processing module is specifically used to: for any text data segment, determine the classification condition that the text data segment first meets according to the priorities of multiple classification conditions contained in the classification method corresponding to the data item to which the text data to be processed belongs, and use the classification condition that is first met as the classification processing result of the text data segment; The device also includes: A display dimension acquisition module is used to acquire all information display dimensions of multiple objects to be processed; a dimension commonality determination module, configured to perform dimension commonality detection on at least one information display dimension among all the information display dimensions according to the content of the at least one information display dimension of the plurality of objects to be processed, so as to determine whether the at least one information display dimension is an information display dimension having dimension commonality; A dimension content updating module is used to update the content of any information display dimension with dimensional commonality using the commonality tag of the information display dimension, wherein any information display dimension with dimensional commonality has at least one commonality tag; A media unit generating module, used to generate a plurality of multimedia units by using all the information display dimensions and their contents, wherein one information display dimension corresponds to different multimedia units according to its different commonality tags, and all the information display dimensions and their contents include: information display dimensions with dimensional commonality and their updated contents; The synthetic media file module is used to select multimedia units belonging to any object to be processed from the multimedia units according to all information display dimensions and contents contained in the object to be processed, and synthesize all selected multimedia units into a multimedia file of the object to be processed.

10. The device according to claim 9, characterized in that The data item to which the text data to be processed belongs is used as the information display dimension of the object to be processed, and the valid text data of the text data to be processed is used as the content of the information display dimension of the object to be processed.

11. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute a computer program stored in the memory, and when the computer program is executed, implement the method described in any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

Images