Standard text cleaning method and device

Abstract

The invention provides a standard text cleaning method and device. The method comprises the following steps: acquiring a standard text; performing cleaning operation on the standard text to obtain a cleaned standard text; the cleaning operation comprises at least one of format normalization processing and semantic consistency processing. By means of the method, efficient and accurate cleaning of the standard text is achieved, and the cleaned standard text is more professional and normative.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a standard text cleaning method and device. Background Art

[0002] Standards, as the technical underpinning of economic activities and social development, are a crucial aspect of a nation's fundamental institutions. Standardization plays a fundamental and guiding role in advancing the modernization of the nation's governance system and capacity. Standard text data mining technology extracts useful information and knowledge from large amounts of unstructured standard text data, enabling data recognition and intelligent retrieval applications. However, due to the diverse multi-dimensional classification of standards and the diversity and complexity of data sources, data often contains significant noise, redundancy, and inconsistencies, which severely impacts data quality. Therefore, standard text cleaning has become a critical step in obtaining high-quality data.

[0003] Related technologies primarily focus on cleaning general text data, but often lack effective cleaning methods for specific domain text data, such as national, industry, and group standards. Standard texts are highly specialized and standardized, and directly applying general text data cleaning methods can result in poor cleaning results and even introduce new errors. Summary of the Invention

[0004] The present invention provides a method and device for cleaning standard texts, which achieve efficient and accurate cleaning of standard texts, making the cleaned standard texts more professional and standardized.

[0005] The present invention provides a standard text cleaning method, which includes the following steps.

[0006] Get standard text; A cleaning operation is performed on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0007] According to a standard text cleaning method provided by the present invention, the format normalization process includes: Identifying formatting irregularities in standard text; the formatting irregularities include at least one of the following: character encoding errors and text misalignment; The format irregularity problem is handled based on preset rules and machine learning.

[0008] According to a standard text cleaning method provided by the present invention, the semantic consistency processing includes: Identifying semantic consistency issues in standard texts; wherein the semantic consistency issues include at least one of the following: inconsistent terminology and inconsistent expression; Addressing semantic consistency issues in the standard text.

[0009] According to a standard text cleaning method provided by the present invention, the semantic consistency processing further includes: For fixed-format text in standard text, templated data cleaning is performed.

[0010] According to a standard text cleaning method provided by the present invention, the cleaning operation further includes: Based on regular expressions, meaningless characters and special symbols in specimen text are removed; Based on the preset adversarial network method, noise information in standard text is eliminated.

[0011] According to a standard text cleaning method provided by the present invention, before performing the cleaning operation on the standard text, the method further includes: The standard text is preprocessed and semantically analyzed; the preprocessing includes at least one of the following: word segmentation, part-of-speech tagging, entity recognition, relationship extraction, and semantic role tagging of the standard text.

[0012] The present invention also provides a standard text cleaning device, comprising the following modules: Acquisition module, used to obtain standard text; The cleaning module is used to perform a cleaning operation on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the standard text cleaning method described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the standard text cleaning methods described above.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned standard text cleaning methods.

[0016] The standard text cleaning method and device provided by the present invention add format normalization processing and semantic consistency processing to the standard text during the cleaning process of the standard text, effectively improves the cleaning effect of the standard text, and realizes efficient and accurate cleaning of the standard text, thereby making the cleaned standard text more professional and standardized. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of the standard text cleaning method provided by the present invention.

[0019] Figure 2 A schematic structural diagram of a standard text cleaning device provided by the present invention.

[0020] Figure 3 A schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0022] The following combination Figure 1-Figure 3 Describe the cleaning method and apparatus of the standard text of the present invention.

[0023] In order to facilitate a clearer understanding of the technical solutions of the various embodiments of the present application, some technical contents related to the various embodiments of the present application are first introduced.

[0024] Existing text data cleaning methods mainly include rule-based methods and machine learning-based methods. Rule-based methods mainly use predefined rules to clean text data. For example, regular expressions can be used to remove noise characters in the text, and dictionary matching can be used to correct incorrect words in the text, and so on. This method is simple and easy to implement, but requires a lot of rule formulation and maintenance, and is difficult to adapt to complex text data. Machine learning-based methods mainly clean text data by training machine learning models. For example, a text classification model can be trained by annotating a large amount of training data to achieve text denoising, deduplication, correction, and supplementation. This method can automatically learn and adapt to complex text data, but requires a large amount of annotated data and computing resources, and the performance of the training model is affected by data quality and model complexity. In addition, some methods will combine manual annotation to further clean and verify the data.

[0025] Existing data cleaning typically involves steps such as data denoising, data deduplication, data correction, and data supplementation. Data denoising primarily removes noise information from the data, such as outliers and missing values; data deduplication primarily removes duplicate information; data correction primarily corrects erroneous information; and data supplementation primarily replaces missing information. Furthermore, with the continuous advancement of natural language processing technology, text data is becoming increasingly important in data cleaning. Natural language processing technology can help understand and process human language, thereby enabling automated cleaning of standard text data.

[0026] Existing text data cleaning methods have solved the problem of text data quality to a certain extent, but there are still some problems and limitations.

[0027] First, existing methods primarily target general text data, but often lack effective cleaning methods for domain-specific text data, such as national standards, industry standards, and group standards. These text data are highly specialized and standardized, and directly applying general text data cleaning methods can lead to poor cleaning results and even introduce new errors.

[0028] Secondly, existing methods mainly focus on basic cleaning tasks such as denoising, deduplication, correction and supplementation of text data. However, there is often a lack of in-depth research and effective solutions on how to further improve the quality of cleaned text data, such as text format normalization and text semantic consistency.

[0029] Finally, existing methods primarily rely on rules or machine learning models, which require extensive rule development or training data and are susceptible to data quality and model complexity. Poor data quality or insufficient model complexity can lead to poor cleaning results. Therefore, developing an efficient, adaptive, and standardized text data cleaning method is a key challenge facing current technology.

[0030] Figure 1 This is one of the flow charts of the standard text cleaning method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Obtain standard text.

[0031] Specifically, related technologies primarily focus on cleaning general text data, but often lack effective cleaning methods for domain-specific text data. Standard text is highly specialized and standardized, and directly applying general text data cleaning methods can lead to poor cleaning results and even introduce new errors.

[0032] In order to solve the above problem, the present application first obtains the standard text to be cleaned. Optionally, the standard text to be cleaned can be a national standard text, an industry standard text, or a group standard text, etc., which is not specifically limited in the present application embodiment.

[0033] Step 102: Perform a cleaning operation on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0034] Specifically, standard texts are highly professional and standardized, and often have high requirements for format standardization. Moreover, existing data cleaning methods mainly focus on basic cleaning tasks such as deduplication, correction and supplementation of text data, while ignoring the problem of semantic consistency in the process of data cleaning. For example, different terms are used to express the same object in the standard text, which greatly affects the professionalism and standardization of the standard text. Therefore, in order to effectively improve the data quality of the standard text, the embodiment of the present application adds format normalization processing and semantic consistency processing to the standard text during the cleaning operation of the standard text, effectively improving the cleaning effect of the standard text, and realizing efficient and accurate cleaning of the standard text, thereby making the cleaned standard text more professional and standardized.

[0035] The method of the above embodiment adds format normalization and semantic consistency processing to the standard text during the cleaning process of the standard text, effectively improves the cleaning effect of the standard text, and realizes efficient and accurate cleaning of the standard text, thereby making the cleaned standard text more professional and standardized.

[0036] In the embodiment, the format normalization process includes: Identify formatting irregularities in standard text; formatting irregularities include at least one of the following: incorrect character encoding and misaligned text; Based on preset rules and machine learning, format irregularities are handled.

[0037] Specifically, in order to improve the professionalism and standardization of the standard text, the embodiment of the present application first identifies the formatting irregularities in the standard text, such as identifying the existence of character encoding errors and text misalignment in the standard text, and adopts a hybrid method based on preset rules and machine learning to correct them, so that the text in the standard text is strictly aligned and the erroneous characters in the standard text are modified in a timely manner, thereby achieving the format standardization processing of the standard text and effectively improving the format standardization degree of the standard text. Optionally, the formatting irregularities in the standard text can be identified based on natural language processing technology, or based on a large model after training, which is not specifically limited in the embodiment of the present application. Optionally, the preset rules can be the rules in the editing process of the standard text, or they can be the default rules in the industry, which is not specifically limited in the embodiment of the present application. Optionally, the model can be trained based on supervised training, so that the trained model can effectively identify the formatting irregularities in the standard text and correct them, thereby improving the efficiency and accuracy of standard text cleaning.

[0038] The method of the above embodiment identifies character encoding errors and text misalignment problems in the standard text, and then uses a hybrid method based on preset rules and machine learning to correct them, so that the text in the standard text is strictly aligned and the erroneous characters in the standard text are modified in a timely manner, thereby realizing the format standardization processing of the standard text and effectively improving the format standardization degree of the standard text.

[0039] In one embodiment, semantic consistency processing includes: Identify semantic consistency issues in standard texts; semantic consistency issues include at least one of the following: inconsistent terminology and inconsistent presentation; Deal with semantic consistency issues in standard texts.

[0040] Specifically, the standard text is highly professional and standardized. In the embodiments of the present application, by identifying and correcting semantic consistency problems in the standard text, such as identifying terminology inconsistencies and expression inconsistencies in the standard text, the cleaning effect of the standard text is effectively improved, and efficient and accurate cleaning of the standard text is achieved, thereby making the cleaned standard text more professional and standardized. Optionally, the terminology inconsistency problem, that is, the use of different terms to express the same object in the standard text, greatly affects the professionalism and standardization of the standard text. The expression inconsistency problem, that is, for the same object, there are contradictory or conflicting expressions in different positions in the standard text, which needs to be corrected to ensure the professionalism and standardization of the standard text. Optionally, the identified semantic consistency problems can be corrected based on preset rules, or the model can be trained based on supervised training, so that the trained model can effectively identify the semantic consistency problems in the standard text and correct them, thereby improving the efficiency and accuracy of standard text cleaning.

[0041] The method of the above embodiment effectively improves the cleaning effect of the standard text by identifying and correcting semantic consistency problems in the standard text, such as identifying inconsistent terminology and expression in the standard text, and realizes efficient and accurate cleaning of the standard text, thereby making the cleaned standard text more professional and standardized.

[0042] In one embodiment, the semantic consistency processing further includes: For fixed-format text in standard text, templated data cleaning is performed.

[0043] Specifically, the semantic consistency processing in the embodiment of the present application also includes templated data cleaning of the fixed format text in the standard text. That is, the standard text is often highly professional and standardized, so the fixed format text in the standard text can be templated. For example, the cover name, English name, release date, implementation date, ICS number, CCS number and other fixed contents of the standard text can be identified. If some of the above fixed contents are inconsistent with the preset template, the preset template can be used to make corrections, thereby achieving the effect of templated data cleaning for the fixed format text in the standard text, accurately and efficiently achieving the semantic consistency processing of the standard text, and improving the cleaning effect of the standard text.

[0044] The method of the above embodiment effectively improves the cleaning effect of the standard text by template processing the fixed format text in the standard text, and realizes efficient and accurate cleaning of the standard text, thereby making the cleaned standard text more professional and standardized.

[0045] In one embodiment, the cleaning operation further comprises: Based on regular expressions, meaningless characters and special symbols in specimen text are removed; Based on the preset adversarial network method, noise information in standard text is eliminated.

[0046] Specifically, the standard text may include many special formulas, symbols, line breaks, etc. In the embodiment of the present application, meaningless characters and special symbols in the standard text are removed based on regular expressions, thereby achieving efficient and accurate cleaning of the standard text, thereby making the cleaned standard text more professional and standardized.

[0047] Optionally, embodiments of the present application also utilize a pre-set adversarial network approach to remove noise information from the standard text. Specifically, adversarial learning between the generator and the discriminator can be used to continuously optimize the model to obtain accurate cleaning results. Optionally, the noise information can include outliers and missing values.

[0048] The method of the above embodiment achieves efficient and accurate cleaning of the standard text by removing meaningless characters and special symbols in the standard text and eliminating noise information in the standard text, thereby making the cleaned standard text more professional and standardized.

[0049] In one embodiment, before performing the cleaning operation on the standard text, the method further includes: Perform preprocessing and semantic analysis on standard text; preprocessing includes at least one of the following: word segmentation, part-of-speech tagging, entity recognition, relationship extraction, and semantic role labeling of standard text.

[0050] Specifically, before cleaning the standard text, the embodiment of the present application first preprocesses and performs semantic analysis on the standard text; wherein the preprocessing includes at least one of the following: word segmentation, part-of-speech tagging, entity recognition, relationship extraction, and semantic role labeling of the standard text; and semantic processing includes semantic analysis and understanding. That is, by preprocessing and semantic analysis of the standard text, on the one hand, the efficiency and accuracy of standard text cleaning can be improved based on the word segmentation, part-of-speech tagging, entity recognition, relationship extraction, and semantic role labeling information of the standard text; on the other hand, based on the semantic analysis of the standard text, an in-depth understanding of the content and structure of the standard text data can be achieved, thereby improving the pertinence and accuracy of standard text cleaning.

[0051] For example, this application provides a standard text cleaning method, the specific process is as follows: 1. Standard text collection. First, collect a large amount of unstructured standard text data. Optionally, standard text data can come from a variety of sources, such as national standards, industry standards, and group standards. The scale of standard text can be adjusted according to actual needs.

[0052] 2. Standard text preprocessing: After collecting standard text PDF data, it is necessary to perform unified OCR preprocessing on the standard text to facilitate subsequent structured processing.

[0053] 3. Data structuring. After data preprocessing is completed, a pre-trained deep learning model, such as the BERT model, is used to structure the data. Taking the national standard text as an example, a domain-adaptive text data preprocessing method is first adopted, including steps such as word segmentation, part-of-speech tagging, and named entity recognition. Specifically, word segmentation adopts a hybrid method based on a dictionary and a statistical model, part-of-speech tagging adopts a conditional random field model, and named entity recognition adopts a deep learning model. Optionally, the data after standard text preprocessing can be input into the BERT model, and the text data can be converted into a structured data form through automatic learning and extraction of the model.

[0054] 4. Data cleaning: Data cleaning mainly removes noise and outliers in the data to improve the quality of the data.

[0055] Template: Data extraction is performed on fixed-format texts in standard texts, such as the standard name of the text cover, English name, release date, implementation date, ICS number, CCS number and other fixed contents for template data cleaning.

[0056] Noise removal: Use regular expressions and other methods to remove meaningless characters and special symbols from the text. In other words, standard text contains many special formulas, symbols, line breaks, etc., which need to be removed for data use.

[0057] Text standardization: This includes lemmatization, synonym replacement, and stop word removal. In standard texts, paragraphs are often incoherent, requiring standardization of specific chapter data, such as terminology and key indicators.

[0058] Word segmentation: Segment text into words, supporting multiple languages such as Chinese and English.

[0059] Data standardization converts data into a unified format for easier processing. After data generation, automated data cleaning techniques are used to remove noise, duplicates, and errors. This step aims to reduce manual intervention and avoid introducing new errors.

[0060] 5. Noise and interference elimination. Low-quality standard text data is eliminated through automated scripts or manual review. Text data with high relevance to the topic is selected as needed. For data containing noise and interference, noise and interference elimination techniques based on adversarial networks are employed. Specifically, adversarial learning between the generator and the discriminator is used to continuously optimize the model for accurate results. In practical applications, the above steps can be modified and adjusted as needed.

[0061] 6. Structured storage: The cleaned, screened, and annotated corpus is stored in a database for subsequent access and use. Optionally, data can be backed up regularly to ensure data security and integrity. That is, the standardized text cleaning method in the embodiment of the present application generates a high-quality, structured corpus through a series of cleaning, screening, and annotation steps to support the training and testing of NLP models. Optionally, the cleaning strategy and parameters can be automatically adjusted according to the characteristics of text data in different fields to improve the efficiency and effectiveness of data cleaning.

[0062] Compared with the existing technology, the standard text cleaning method of the above embodiment has the following beneficial effects: (1) This application performs structured processing on unstructured data such as national standard texts, industry standard texts, and group standard texts, and then performs data cleaning, which effectively solves the problem that the existing methods are mainly for general text data and lack effective cleaning methods for text data in specific fields. Through this targeted cleaning method, the quality of text data in specific fields can be greatly improved, ensuring that the cleaning effect is more accurate and effective. (2) Compared with the existing technology, the technical solution of this application does not require a large amount of manual intervention in the data cleaning process, which not only saves time and energy, but also avoids the introduction of new errors due to human factors. (3) The technical solution of this application not only focuses on basic cleaning tasks such as denoising, deduplication, correction and supplementation of text data, but also conducts in-depth research on how to further improve the quality of standard text data after cleaning, such as text format normalization and text semantic consistency. This makes up for the shortcomings of the existing methods in improving the quality of text data, makes the cleaned text data more standardized and consistent, and improves the readability and usability of the data. Furthermore, the standard text cleaning method in this application can automatically learn and adapt to complex standard text data, greatly improving the efficiency and accuracy of standard text data cleaning. Furthermore, this method can be customized according to actual needs, making it more flexible and applicable to a variety of different standard text data cleaning scenarios.

[0063] The standard text cleaning device provided by the present invention is described below. The standard text cleaning device described below and the standard text cleaning method described above can be referred to each other. Figure 2Shown, including: An acquisition module 210 is used to acquire a standard text; The cleaning module 220 is used to perform a cleaning operation on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0064] Optionally, the cleaning module 220 is specifically configured to identify format irregularities in the standard text; format irregularities include at least one of the following: character encoding errors and text misalignment; Handle format irregularities based on preset rules and machine learning Optionally, the cleaning module 220 is specifically configured to identify semantic consistency issues in the standard text; semantic consistency issues include at least one of the following: inconsistent terminology and inconsistent expression; Deal with semantic consistency issues in standard texts.

[0065] Optionally, the cleaning module 220 is specifically configured to perform templated data cleaning on fixed-format text in the standard text.

[0066] Optionally, the cleaning module 220 is further configured to remove meaningless characters and special symbols from the specimen text based on a regular expression method; Based on the preset adversarial network method, noise information in standard text is eliminated.

[0067] Optionally, the cleaning module 220 is further used to perform preprocessing and semantic analysis on the standard text; the preprocessing includes at least one of the following: word segmentation, part-of-speech tagging, entity recognition, relationship extraction and semantic role tagging of the standard text.

[0068] Figure 3 The following illustrates a physical structure diagram of an electronic device, which may include a processor 310, a communications interface 320, a memory 330, and a communications bus 340. The processor 310, the communications interface 320, and the memory 330 communicate with each other via the communications bus 340. The processor 310 may invoke logic instructions in the memory 330 to execute a standard text cleaning method. The method includes: obtaining standard text; performing a cleaning operation on the standard text to obtain the cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0069] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0070] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the standard text cleaning method provided by the above methods, which includes: obtaining standard text; performing a cleaning operation on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0071] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the standard text cleaning method provided by the above methods, the method comprising: obtaining a standard text; performing a cleaning operation on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0073] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A standard text cleaning method, characterized in that: include: Get standard text; A cleaning operation is performed on the standard text to obtain a cleaned standard text; the cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

2. The method for cleaning standard text according to claim 1, characterized in that: The format normalization process includes: Identifying formatting irregularities in standard text; the formatting irregularities include at least one of the following: character encoding errors and text misalignment; The format irregularity problem is handled based on preset rules and machine learning.

3. The method for cleaning standard text according to claim 1, characterized in that: The semantic consistency processing includes: Identifying semantic consistency issues in standard texts; wherein the semantic consistency issues include at least one of the following: inconsistent terminology and inconsistent expression; Addressing semantic consistency issues in the standard text.

4. The method for cleaning standard text according to claim 3, characterized in that: The semantic consistency processing further includes: For fixed-format text in standard text, templated data cleaning is performed.

5. The method for cleaning a standard text according to any one of claims 1 to 4, characterized in that: The cleaning operation further comprises: Based on regular expressions, meaningless characters and special symbols in specimen text are removed; Based on the preset adversarial network method, noise information in standard text is eliminated.

6. The method for cleaning standard text according to claim 5, characterized in that: Before performing the cleaning operation on the standard text, the method further includes: The standard text is preprocessed and semantically analyzed; the preprocessing includes at least one of the following: word segmentation, part-of-speech tagging, entity recognition, relationship extraction, and semantic role tagging of the standard text.

7. A standard text cleaning device, characterized in that: include: Acquisition module, used to obtain standard text; A cleaning module, configured to perform a cleaning operation on the standard text to obtain a cleaned standard text; The cleaning operation includes at least one of the following: format normalization processing and semantic consistency processing.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the standard text cleaning method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the standard text cleaning method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the standard text cleaning method according to any one of claims 1 to 6 is implemented.

Images