Method, system, equipment and medium for checking words in certificate of Chinese dictionary
Through automated preprocessing and intelligent retrieval methods, the problems of low efficiency and high error rate in traditional manual editing and proofreading have been solved, the efficiency and accuracy of Chinese dictionary and bibliographic evidence checking have been achieved, the false alarm rate has been reduced, and the intelligent level of dictionary compilation has been improved.
Patent Information
- Application Number
- CN202510945848.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-09
- Publication Date
- 2025-10-10
AI Technical Summary
The efficiency of word checking in traditional manually edited Chinese dictionaries is low and the error rate is high, making it difficult to ensure the consistency between the textual evidence and the word entries.
Automated preprocessing and intelligent retrieval methods are used to conduct multi-version word searches using standardized symbols, dynasty names, and author names, and non-biographical texts are filtered based on keywords, with unmatched results automatically marked.
It has significantly improved the efficiency and accuracy of Chinese dictionary bibliographical evidence checking, reduced the false alarm rate, reduced the workload of manual proofreading, and ensured the rigor of dictionary compilation and the intelligent upgrade of the process.
Smart Images
Figure CN120763345A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a language dictionary evidence word checking method, system, device and medium, belonging to the technical field of dictionary retrieval. BACKGROUND
[0002] The evidence in the language dictionary is the strongest proof of the establishment of the word entry, which can reflect the meaning, usage, style color, emotional color and grammatical features of the word entry. If the evidence time is properly selected, it can also trace the birth, evolution and even extinction process of the word entry. Therefore, if it is detected that the evidence does not contain complete and accurate word entries, it means that there may be word entry errors or evidence problems in the word entry, which directly affects the quality of the word entry and even the storage and deletion.
[0003] However, in the traditional compilation process, manual editing often cannot efficiently ensure the consistency of the evidence and the word entry due to problems such as character shape confusion, non-standard character usage, or format omissions. SUMMARY
[0004] Therefore, the present application provides a language dictionary evidence word checking method, system, device and medium to solve the problems of low efficiency and high error rate of manual correction.
[0005] The first object of the present application is to provide a language dictionary evidence word checking method.
[0006] The second object of the present application is to provide a language dictionary evidence word checking system.
[0007] The third object of the present application is to provide an electronic device.
[0008] The fourth object of the present application is to provide a storage medium.
[0009] The first object of the present application can be achieved by adopting the following technical solution: A language dictionary evidence word checking method, the method comprising: Obtaining word entries, pinyin, literature original text and segmented evidence content from preprocessed literature data; Standardizing the dynasty name, author name and literature name by replacing and / or removing redundant symbols; Searching for word entries in the evidence content and marking unmatched results; Performing multi-version retrieval of the word entries in traditional or simplified form according to the dictionary type; Filtering non-evidence text based on predefined keywords and removing false positive results; Outputting the remaining unmatched word entries and associated data as a review result.
[0010] Further, the searching for the word in the documentary content, marking the non-matching result, comprises: locating the word in the documentary content by string matching algorithm, and marking as missing or wrong if not matched.
[0011] Further, the standardizing the dynasty name, author name and document name by replacing and / or removing redundant symbols comprises a first operation for standardizing the document name and a second operation for standardizing the dynasty name and author name. The redundant symbols involved in the first operation comprise at least one of “《”, “》”, and “*”. The second operation comprises: replacing “《*” with null value; selecting the target column, emptying the data row containing “:”, and then canceling the selection state; selecting the target column, emptying the data row containing “”, and then canceling the selection state; selecting the target column, emptying the data row containing “, ”, and then canceling the selection state; selecting the target column, emptying the data row containing “.”, and then canceling the selection state.
[0012] Further, the multi-version searching for the word in the form of traditional or simplified Chinese according to the type of dictionary comprises: if the dictionary is ancient literature, searching for the word in the form of traditional Chinese; if the dictionary is modern literature, searching for the word in the form of simplified Chinese; if the dictionary is pre-set literature, searching for the word in both the form of traditional Chinese and the form of simplified Chinese.
[0013] Further, the method further comprises: extracting heterographic words from the original text of the document, the heterographic words being identified by pre-defined keywords; segmenting the heterographic words into independent words, and searching for the heterographic words in the documentary content, and clearing the corresponding error report if matched.
[0014] Further, before outputting the remaining unmatched words and associated data as the review result, the method further comprises: selecting the data row containing the key word of etymology in the first documentary and clearing the error report.
[0015] Further, the pre-processing of the document data comprises: based on the language dictionary data, extracting the first pure explanatory sentence according to the position of the first book name; based on the language dictionary data, selecting the sentence with the explanation in front and the document behind according to the relative position of the period and the book name, and extracting the second pure explanatory sentence and the first pure document name. Based on the language dictionary data, a second pure document name is extracted according to the first book name position and the relative position of the period and the book name; Based on the language dictionary data, dynasty author information is extracted; Based on the language dictionary data, a to-be-cut example sentence is extracted and cut into a single sentence.
[0016] The second object of the present application can be achieved by adopting the following technical solution: A language dictionary book evidence word checking system, the system comprises: An acquisition unit is configured to acquire words, pinyin, literature original text and cut book evidence content from preprocessed literature data; A standardization unit is configured to standardize dynasty names, author names and document names by replacing and / or removing redundant symbols; A marking unit is configured to search for words in the book evidence content and mark unmatched results; A searching unit is configured to search for words in multiple versions of traditional or simplified Chinese according to the type of the dictionary; A cleaning unit is configured to clean false positive results based on pre-defined keywords to filter non-book evidence text; An output unit is configured to output remaining unmatched words and associated data as a review result.
[0017] The third object of the present application can be achieved by adopting the following technical solution: An electronic device comprises a processor and a memory for storing a processor-executable program, when the processor executes the program stored in the memory, the language dictionary book evidence word checking method described above is realized.
[0018] The fourth object of the present application can be achieved by adopting the following technical solution: A storage medium stores a program, when the program is executed by a processor, the language dictionary book evidence word checking method described above is realized.
[0019] The embodiments of the present application have the following beneficial effects relative to the prior art: The embodiment of the present application improves the efficiency and accuracy of the Chinese dictionary evidence checking by automatic preprocessing and intelligent retrieval. The beneficial effects include: (1) standardized processing: unified symbols, dynasty names and author names format, eliminate data noise, ensure consistency of retrieval; (2) multi-dimensional matching: support multi-version word entry retrieval of simplified and traditional Chinese, adapt to different types of dictionary specification requirements; (3) accurate filtering: based on keyword library to filter out non-evidence text, effectively reduce the false positive rate; (4) efficient output: automatic marking of unmatched word entries and association of original data, greatly reducing the workload of manual proofreading, and providing clear evidence for word entry correction or evidence supplement. Overall, the technology not only guarantees the rigor of dictionary compilation, but also realizes the intelligent upgrading of the process. BRIEF DESCRIPTION OF DRAWINGS
[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the structures shown in the drawings.
[0021] Figure 1 The flow chart of the Chinese dictionary evidence word entry checking method of the embodiment 1 of the present application.
[0022] Figure 2 The flow chart of the special-shaped word removal method of the embodiment 1 of the present application.
[0023] Figure 3 The structural block diagram of the Chinese dictionary evidence word entry checking system of the embodiment 2 of the present application.
[0024] Figure 4 The structural block diagram of the electronic device of the embodiment 3 of the present application. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme of the embodiments of the present application will be described clearly and completely in the following with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0026] Embodiment 1: as shown in the present embodiment provides a Chinese dictionary evidence word entry checking method, the method comprises the following steps: Figure 1 S101 obtain the word, pinyin, original text and split book evidence content from the preprocessed literature data.
[0027] Further, the preprocessed literature data includes: Based on the Chinese dictionary data, the first pure explanatory sentence is extracted according to the position of the first book name.
[0028] Based on the Chinese dictionary data, the second pure explanatory sentence and the first pure literature name are extracted according to the relative position of the period and the book name.
[0029] Based on the Chinese dictionary data, the second pure literature name is extracted according to the position of the first book name and the relative position of the period and the book name.
[0030] Based on the Chinese dictionary data, the dynasty and author information is extracted.
[0031] Based on the Chinese dictionary data, the example sentence to be split is extracted and split into single sentences.
[0032] It can be understood that data preprocessing is equivalent to splitting the word into 5 blocks of word, pure explanatory sentence, dynasty name and author name, literature name, and book evidence sentence. Finally, the example sentence is split into single sentences.
[0033] It should be noted that the word in the automatic check of Chinese dictionary book evidence is only split into 3 blocks of word, pure explanatory sentence, book evidence (including dynasty name and author name, literature name, and book evidence sentence). However, in order to analyze and detect the Chinese dictionary comprehensively, the word in the Chinese dictionary is divided into 5 blocks at one time in the "data preprocessing" stage, which is the most economical and efficient method.
[0034] (1) The first time the word is divided, so that each fragment has at most one book evidence (including dynasty name and author name, literature name, and book evidence sentence).
[0035] Note: The specific data and program process files for the following program description are two: File 1: "Special Topic 1-8 Pairs of Punctuation - Special Topic 10 Analogical Simplified Characters"; File 2: "Special Topic 9-10 Program and Sample".
[0036] 1. Divide the Chinese dictionary into 4 columns.
[0037] In file 1: "Special Topic 1-8 Pairs of Punctuation - Special Topic 10 Analogical Simplified Characters" new table "Data Source"; Copy the contents of the Chinese dictionary manuscript file (Word or Txt format) to "Data Source", and split it into the following 4 columns using replace or function: (1)
Original number (delete blank line)
[0038] (2)
Word
Pinyin of word
Explanation
[0039] Note: Unexpected line breaks cause the word in the
Word
Word
[0040] Processing or cleaning method: Use the length function =LEN(B2) to calculate the length of the B column string and arrange it in descending order. The length of the word in the B column string is particularly large, which may have incorrectly extracted the interpretation sentence as the word, and manual intervention is required.
[0041] 2. Add a split marker @ at the end of each document; Replace the data in the
Explanation
[0042] Add a new column
Number of documents
Explanation operation
[0043] 3. Split the original document interpretation (including interpretation and evidence) with @ as the split marker, so that each piece can have at most one piece of evidence (evidence includes: dynasty, author, document name, document sentence).
[0044] Select the entire data, arrange it in descending order according to the operation value of
Number of documents
Explanation operation
Document split
Document split
[0045] (2) Merge and align the evidence of each word entry in the
Data source
[0046] Note: The number of book evidence provided for each word entry in the Chinese dictionary varies. For the sake of convenience, the "Data Source" should be processed into the following neat columns: (1)
Original number (remove blank lines)
Word
Pinyin of the word
Explanation
Cut literature unique serial number
Cut literature
[0047] Assuming the first word has n book evidence, the above 6 columns will repeat n rows, and the
Cut literature unique serial number
[0048] Take the 3 columns of data in the above 6 columns to create a new table "Literature Cutting Results", which is ready for the next step. These 3 columns of data are: (1)
Original number (remove blank lines)
Cut literature unique serial number
Cut literature
[0049] (Three) Further cut the C column in "Literature Cutting Results" into 4 fields: pure interpretation sentence, dynasty name + author name, pure literature name, and example sentence to be cut.
[0050] 1. Extract the first batch of pure interpretation sentences (without book evidence).
[0051] In the file "Literature Cutting Results", create a new D column
first 《position
first position
period position minus book name position
[0052] Select all columns of data, sort the results in ascending order by the first 《position in column D, and the rows with error value "#VALUE!" will all be at the end. They are all pure interpretation sentences (Note: Because pure interpretation sentences have no literature, the FIND function to find the book name will report an error). Copy them and create a new table "Pure Interpretation Sentences" document for future use. This file contains 3 columns of data: Column A:
Original document unique number
Split document unique serial number
Pure interpretation sentence
[0053] After the above operation, in the file "Document Split Results", clear or ignore these pure interpretation sentences (temporarily leave them for technical personnel reference, but do not participate in the following operation).
[0054] 2. Extract pure document name and the second batch of pure interpretation sentences (not including book evidence).
[0055] Select all columns of the file "Document Split Results", and sort them in order according to the value of column F
period position minus book name position
[0056] Divide these data into two parts again, (1) pure interpretation sentence; (2) dynasty + author name + pure document name + example sentence.
[0057] Operation steps are: Copy the data of column C
split document
pure document name
[0058] In column G, replace "》*" with "》" (Note: the range of operation rows is: data rows with negative F column data).
[0059] The result obtained in the previous step is: pure interpretation sentence + dynasty + author name + pure document name.
[0060] Copy the data of column G participating in the operation range after the above replacement to column H, and name the column as
the second batch of pure interpretation sentences
[0061] In column H
the second batch of pure interpretation sentences
pure interpretation sentence + dynasty + author
[0062] Note: Some document names such as "Shiji" do not have "dynasty + author name" according to the style of some dictionaries, so this step only leaves pure interpretation sentences.
[0063] Copy column H obtained in the previous step once to a new column-I column, and name the column as:
dynasty name + author name
Original unique number
Split document unique serial number
Pure paraphrased sentence
[0064] 3, extract the second batch of "pure document name", and the first and second batch of "dynasty name + author name".
[0065] In column I
dynasty name + author name
[0066] In the file "document segmentation results", select all data, and filter the following two columns at the same time: Column D
first 《position
period position minus book name position
[0067] After filtering the above two columns at the same time, replace "*《" with "《" in column G
pure document name
pure document name
[0068] Note: Regarding the second batch of "pure document name" data, because there are sometimes two or more document names in a single cell, the simple operation of replacing "*《" with "《" and then replacing "》*" with "》" will only leave the last pure document name. The remnants of the previous several document names are: 《《, or 《《《, 《《《《, etc. - Handle later in the upgrade process.
[0069] After filtering the above two columns at the same time, input the function: =C column in column I
dynasty name + author name
dynasty name + author name
[0070] Replace "《*" with: null, and finally get the second batch of "dynasty name + author name" data in the original position of column I.
[0071] 4, extract "to be segmented example sentences" (pure example sentences) Copy column C
segmented document
to be segmented example sentences
[0072] The result of the above step is "pure example sentence", named:
to be cut example sentence
[0073] (Four) Cut the J column
to be cut example sentence
[0074] Select the following 3 columns of data from the "document segmentation results" file created in the previous step, and create a new file "example sentence segmentation data source": New column A:
original manuscript unique number
segmentation document unique serial number
to be cut example sentence
[0075] Take the newly created file "example sentence segmentation data source" as "data source", and cut the new column C:
to be cut example sentence
[0076] When operating the "segmentation and merging program" again (the function template used when cutting the book evidence above, which can be copied here to cut, merge, and align single sentences), pay attention to: (1) Change the file name parameters in the program to: example sentence segmentation data source; (2) Add column 6
segmentation document unique serial number
segmentation document unique serial number
segmentation document unique serial number
:,;?!……
。
。
[0077] Finally, filter column 1 of the file "example sentence segmentation function" to remove 0 and empty values, and copy the results selectively and paste them as "numeric values" into the newly created file "example sentence segmentation results". The file "example sentence segmentation results" has 4 columns of data: Column A:
original manuscript unique number
Split literature unique serial number
Split example sentence unique serial number
Split example sentence
[0078] In this step, the starting file is the last result of "Literature Split Result" in "(Three) Further split C column in Literature Split Result into four fields: pure interpretation sentence, dynasty name + author name, pure literature name, and example sentence to be split" of "Data Preprocessing" - that is, Literature Split Result in File 1: "Thematic 1-8 Pairs of Punctuation - Thematic 10 Analogical Simplified Characters", select the following data columns from this file and copy them to the newly created file "Thematic 9 Sample (Search for Word in Book Evidence)" as follows: Column A:
Original (remove blank lines) unique serial number
Split literature unique serial number
Word
Pinyin
Original full text data (backup)
Dynasty name + author name
Pure literature name
Pure interpretation sentence
Split literature
Word
Pinyin of word
Original full text data (backup)
Original serial number (remove blank lines)
Word
Pinyin of word
Interpretation
Original serial number (remove blank lines)
[0079] (2) Insert the function into column C
Word
Pinyin of word
Original full text data (backup)
[0080] S102 standardizing the dynasty name, author name and literature name by replacing and / or removing redundant symbols.
[0081] Further, the standardizing the dynasty name, author name and literature name by replacing and / or removing redundant symbols comprises a first operation for standardizing the literature name and a second operation for standardizing the dynasty name and author name. The redundant symbols involved in the first operation comprise at least one of “《”, “>”, and “*”. The second operation comprises: replacing “《*” with null; selecting the target column, emptying the data rows containing “:”, and then canceling the selection state; selecting the target column, emptying the data rows containing “”, and then canceling the selection state; selecting the target column, emptying the data rows containing “, ”, and then canceling the selection state; selecting the target column, emptying the data rows containing “.”, and then canceling the selection state.
[0082] Specifically, the G column:
pure literature name
[0083] The F column:
dynasty name + author name
[0084] S103 Search for the word in the book evidence content, and mark the unmatched result (initial search).
[0085] Further, the searching for the word in the book evidence content and marking the unmatched result comprises: Locating the word in the book evidence content by string matching algorithm, and marking as missing or error if not matched.
[0086] Specifically, a new J column
search for word in book evidence
[0087] Since the result is “#VALUE!” when the above function search fails, the J column is filtered with the condition =#VALUE! Under the above filtering condition, a new K column
book evidence word search result
[0088] As an optional implementation, the following cases also belong to invalid error reporting (need to be deleted): mostly belong to book evidence that explains words. The key words of such book evidence are: Note: Send: Dialectics: Collation: Brief: School: Explanation: Examples, the word does not appear in the book evidence, and the book evidence list is as follows: 1) Wen Yiduo’s “Jiaozhongqing’s Wife” in “Jiaozhongqing’s Wife”: “Ancient music is very good, and the story is also good.” The document name contains
Note:
[0089] S104 Multiple version search for the word in traditional or simplified form according to the type of dictionary.
[0090] Further, the multiple version search for the word in traditional or simplified form according to the type of dictionary comprises: If the dictionary is ancient literature, search for the word in traditional form; If the dictionary is modern literature, search for the word in simplified form; If the dictionary is a preset literature, search for the word in both traditional and simplified forms.
[0091] For example: in the special dictionary "Chinese Dictionary", ancient Chinese book evidence uses traditional Chinese characters, and modern Chinese book evidence uses simplified Chinese characters, so the entry should be in traditional and simplified Chinese characters respectively. Twice search results are "#VALUE!" which are the real problems found in this topic. For other dictionaries, if the book evidence is in traditional Chinese characters, then the " =FIND(C2, I2, 1)" function needs to search for traditional Chinese characters only once. If the book evidence is in simplified Chinese characters, then the " =FIND(C2, I2, 1)" function needs to search for simplified Chinese characters only once.
[0092] S105 filters the non-book evidence text based on the predefined keywords, and removes false positive results.
[0093] In this step, in column I
split literature
:
split literature
split literature
book evidence search entry results
[0094] The strings that need to be screened in column I
split literature
[0095] S106 outputs the remaining unmatched entries and associated data as the review results.
[0096] Further, as shown in Figure 2 , the method further includes: S201 extracts variant characters from the literature original text, and the variant characters are identified by predefined keywords.
[0097] S202 splits the variant characters into independent entries and searches for the variant characters in the book evidence content. If matched, remove the corresponding error results.
[0098] Specifically, many language dictionaries use the form "also known as something" or "also known as something" in their definitions to explain variant forms of a term. Subsequent textual evidence may use either the term itself or its variant form. Therefore, this topic should not report an error whether the documented evidence contains the term itself or its variant form.
[0099] The operations for processing variant word entries are: 1. Copy the data in column E [Full text data of original manuscript (backup)] and selectively paste it as a value in column L. Name column L [Variant adverb 1]. 2. Filter column L [Variant Adverb 1], and clear the rows that do not contain all of the following keywords: [作“][同“][称“][一作][也作][一称][也称][也称]——, and clear the rows that do not contain any of the following keywords: [作“][同“][叫“][一作][也作][也称][也称]——; 3. Filter the L column [Variant Adverb 1], and find the rows containing [作“], and replace [*作“] with a null value; then replace [".*] with a null value.
[0100] 4. Filter the L column [Variant Adverb 1], and replace the data rows containing [Same as "] with [* Same as "]: empty value; then replace [."*] with empty value.
[0101] 5. Filter the L column [Variant Adverb 1], and replace the data rows containing [称“] with [*称“]: null value; then replace [".*] with null value.
[0102] 6. Filter column L [Variant Adverb 1], find the rows containing [一作], replace [*一作] with a null value; then replace [. *] with a null value.
[0103] 7. Filter column L [Variant Adverb 1], and replace the rows containing [Also] with a null value for [*Also]; then replace [. *] with a null value.
[0104] 8. Filter column L [Variant Adverb 1], and replace the rows containing [Also] with a null value for [*Also]; then replace [. *] with a null value.
[0105] 9. Filter column L [Variant Adverb 1], find the rows containing [一称], replace [*一称] with a null value; then replace [. *] with a null value.
[0106] 10. Filter column L [Variant Adverb 1], and replace the rows containing [Also Known As] with a null value for [*Also Known As]; then replace [. *] with a null value.
[0107] 11、Filter L column
Variant word entry 1
Also called
Also called
[0108] 12、Split continuous multiple variant word entries After the above operation, in the result of L column
Variant word entry 1
”“
”、“
”,“
@
@
Variant word entry 1
Variant word entry 2
Variant word entry 3
Variant word entry 1
Variant word entry 2
Find variant word entry 1
Find variant word entry 1
Split literature
Find variant word entry 2
Find variant word entry 1
Split literature
Find word entry in book evidence
Book evidence word entry result
Find variant word entry 1
Book evidence word entry result
Find variant word entry 2
Book evidence word entry result
[0109] Furthermore, before outputting the remaining unmatched entries and associated data as the review results, the method further includes: Filter the data row containing etymological keywords in the first documentary evidence and clear the error.
[0110] Specifically, the first documentary evidence in a language dictionary is sometimes etymological, tracing back to the initial state of word formation. At this time, the documentary evidence may not necessarily contain complete and accurate entries. Therefore, filter the data column containing the first documentary evidence. If it contains the following keywords, clear the error results in column K [Documentary Evidence Query Result]. These keywords are: Speech Language version; subsequent cause; Afterwards; Later said; Later called.
[0111] Finally, after the above multiple data cleanings, the remaining errors in column K [Document Evidence Search Results] are the valid error results for this topic. The following data columns are used as the final review results for this topic: Column A: [Original manuscript (excluding blank lines) unique serial number]; Column B: [Unique serial number of the segmented document]; Column C: [heading]; Column D: [Pinyin]; Column E: [Full text data of original manuscript (backup)]; Column I: [Segmented documents]; Column K [Documentary evidence search results].
[0112] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium.
[0113] It should be noted that although the method operations of the above embodiments are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed to achieve the desired results. Rather, the depicted steps may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.
[0114] Example 2: Figure 3 As shown, this embodiment provides a system for checking entries in a language dictionary or documentary evidence. The system includes an acquisition unit 301, a standardization unit 302, a marking unit 303, a retrieval unit 304, a clearing unit 305, and an output unit 306. The specific functions of each unit are as follows: The acquisition unit 301 is used to acquire the headword, pinyin, original text of the document and the segmented document content from the pre-processed document data; A standardization unit 302 is used to standardize dynasty names, author names, and document names by replacing and / or removing redundant symbols; A marking unit 303 is used to search for terms in the document content and mark unmatched results; Retrieval unit 304, for performing multi-version retrieval of traditional or simplified Chinese entries based on dictionary type; A clearing unit 305 is used to filter non-documentary evidence text based on predefined keywords and clear false positive results; The output unit 306 is used to output the remaining unmatched words and associated data as the review results.
[0115] Example 3: Figure 4 As shown, this embodiment provides an electronic device, which includes a processor 402, a memory, an input device 403, a display device 404, and a network interface 405 connected via a system bus 401. The processor 402 is used to provide computing and control capabilities, and the memory includes a non-volatile storage medium 406 and an internal memory 407. The non-volatile storage medium 406 stores an operating system, a computer program, and a database. The internal memory 407 provides an environment for the operation of the operating system and computer program in the non-volatile storage medium 406. When the computer program is executed by the processor 402, the method for checking entries in a language dictionary document of the first embodiment is implemented as follows: Obtaining the headwords, pinyin, original text and segmented documentary evidence from the pre-processed document data; Standardize dynasty names, author names, and document names by replacing and / or removing redundant symbols; Search for terms in the content of documentary evidence and mark unmatched results; Search for multiple versions of entries in traditional or simplified Chinese according to dictionary type; Filter non-documentary text based on predefined keywords to eliminate false positives; The remaining unmatched terms and associated data are output as the review results.
[0116] Embodiment 4: This embodiment provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the method for checking entries in a language dictionary document of the above embodiment 1 is implemented as follows: Obtaining the headwords, pinyin, original text and segmented documentary evidence from the pre-processed document data; Standardize dynasty names, author names, and document names by replacing and / or removing redundant symbols; In the book content, search for the word, mark the non-matching results; According to the dictionary type, the word is searched in the traditional or simplified form of multiple versions; Based on the pre-defined keywords, the non-book text is filtered, and the false positive results are removed; The remaining unmatched words and associated data are output as the review results.
[0117] It should be noted that the computer readable storage medium of the embodiment can be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium may, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage medium can include, but are not limited to: electrical connection with one or more conductive wires, portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
[0118] In the embodiment, the computer readable storage medium can be any tangible medium containing or storing programs that can be used or combined with instruction execution system, device or device. While in the embodiment, the computer readable signal medium can include data signals propagating in the baseband or as part of a carrier wave, which carries computer readable programs. Such propagating data signals can take various forms, including but not limited to electromagnetic signals, optical signals or any suitable combination of the above. The computer readable signal medium can also be any computer readable storage medium other than the computer readable storage medium, which can send, propagate or transmit programs for use by or in combination with instruction execution system, device or device. The computer program contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to: wire, cable, RF (radio frequency), etc., or any suitable combination of the above.
[0119] The computer readable storage medium described above can be written in one or more programming languages or combinations of languages for execution by computers including object oriented programming languages such as Java, Python, C++ or conventional procedural programming languages such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).
[0120] To sum up, the embodiments of the present application significantly improve the efficiency and accuracy of the Chinese dictionary book evidence checking through automatic preprocessing and intelligent retrieval. The beneficial effects thereof specifically include: (1) standardized processing: through unifying the symbol, dynasty name and author name format, eliminating data noise and ensuring consistency of retrieval; (2) multi-dimensional matching: supporting simplified and traditional Chinese multi-version entry retrieval, adapting to the specification requirements of different types of dictionaries; (3) accurate filtering: based on the keyword library to filter out non-book evidence text, effectively reducing the false positive rate; (4) efficient output: automatically marking unmatched entries and associating original data, greatly reducing the manual proofreading workload, and providing clear evidence for entry correction or book supplement. Overall, the technology not only guarantees the rigor of dictionary compilation, but also realizes the intelligent upgrading of the process.
[0121] The above describes only the preferred embodiments of the present application patent, but the protection scope of the present application patent is not limited to this. Any person skilled in the art can make equivalent substitutions or changes to the technical solutions and inventive concepts of the present application patent within the scope disclosed by the present application patent, which still belongs to the protection scope of the present application patent.
Claims
1. A method for checking entries in a language dictionary, characterized in that: The method comprises: Obtaining the headwords, pinyin, original text and segmented documentary evidence from the pre-processed document data; Standardize dynasty names, author names, and document names by replacing and / or removing redundant symbols; Search for terms in the content of documentary evidence and mark unmatched results; Search for multiple versions of entries in traditional or simplified Chinese according to dictionary type; Filter non-documentary text based on predefined keywords to eliminate false positives; The remaining unmatched terms and associated data are output as the review results.
2. The method for checking entries in a Chinese dictionary according to claim 1, characterized in that: The search for terms in the documentary evidence content and marking of unmatched results include: A string matching algorithm is used to locate the word in the documentary evidence content. If there is no match, it is marked as missing or incorrect.
3. The method for checking entries in a Chinese dictionary according to claim 1, characterized in that: The standardization of dynasty names, author names, and document names by replacing and / or removing redundant symbols includes a first operation and a second operation, wherein the first operation is used to standardize the document name, and the second operation is used to standardize the dynasty name and author name; The redundant symbols involved in the first operation include at least one of ", ", and *; The second operation includes: Replace "《*" with: empty value; Filter the target column, clear the data rows containing colons [:], and then cancel the filtering status; Filter the target column, clear the data rows containing double quotes ["] on the left, and then cancel the filter status; Filter the target column, clear the data rows containing commas [,], and then cancel the filtering status; Filter the target column, clear the data rows containing a period [.], and then cancel the filter status.
4. The method for checking entries in a Chinese dictionary according to claim 1, characterized in that: The multi-version search of the traditional or simplified Chinese entries according to the dictionary type includes: If the dictionary is an ancient document, the traditional Chinese form of the word will be searched; If the dictionary is a modern document, the entries are searched in simplified Chinese; If the dictionary is the default document, the entries will be searched in both traditional and simplified Chinese.
5. The method for checking entries in a Chinese dictionary according to claim 1, characterized in that: The method further comprises: Extracting variant words from the original text of the document, wherein the variant words are identified by predefined keywords; Split variant words into independent entries and search for variant words in the documentary evidence content. If there is a match, clear the corresponding error result.
6. The method for checking entries in a Chinese dictionary according to claim 1, characterized in that: Before outputting the remaining unmatched entries and associated data as the review results, the method further includes: Filter the data row containing etymological keywords in the first documentary evidence and clear the error.
7. The method for checking entries in a Chinese dictionary according to claim 1, characterized in that: Preprocessing of literature data, including: Based on the Chinese dictionary data, extract the first pure interpretation sentence according to the position of the first quotation mark of the book title; Based on the language dictionary data, according to the relative positions of periods and quotation marks, sentences with explanations placed first and references placed second are selected, and the second pure explanation sentence and the first pure reference name are extracted; Based on the language dictionary data, extracting the second pure document title according to the position of the first quotation mark and the relative position of the period and the quotation mark; Extracting dynasty and author information based on the language dictionary data; Based on the language dictionary data, example sentences to be segmented are extracted and segmented into individual sentences.
8. A system for checking entries in Chinese dictionary evidence, characterized by: The system comprises: An acquisition unit is used to acquire the headword, pinyin, original text of the document and the segmented document content from the pre-processed document data; A standardization unit for standardizing dynasty names, author names, and document names by replacing and / or removing redundant symbols; A marking unit is used to search for terms in the content of documentary evidence and mark unmatched results; A search unit for searching multiple versions of entries in traditional or simplified Chinese according to the dictionary type; A cleaning unit is used to filter non-documentary evidence text based on predefined keywords and remove false positive results; The output unit is used to output the remaining unmatched words and related data as the review results.
9. An electronic device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, it implements the method for checking entries in a language dictionary document as described in any one of claims 1 to 7.
10. A storage medium storing a program, characterized in that: When the program is executed by a processor, the method for checking entries in a language dictionary document as described in any one of claims 1 to 7 is implemented.