Text processing method, device, computer equipment and computer readable storage medium

By obtaining the word segmentation results of the input text and matching the pending text from the standard text database, the difference value is calculated to output the matching standard text, which solves the problem of large differences in text processing in the medical field, and improves accuracy and efficiency.

CN114638226BActive Publication Date: 2024-12-06WINNING HEALTH TECHNOLOGY GROUP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210322720.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-29
Publication Date
2024-12-06
Estimated Expiration
2042-03-29

AI Technical Summary

Technical Problem

In the medical field, when processing text in the text in the medical field, due to the possible existence of special symbols and typos in the text to be stored, the difference between the output standard text and the input text is relatively large, the accuracy is low, and manual adjustment is required, which is inefficient.

Method used

By obtaining the word segmentation result of the input text, obtaining the matching pending text from the standard text database based on the word segmentation result, calculating the difference value between the input text and the pending text, and outputting the standard text matching the input text to reduce the difference and improve the accuracy.

Benefits of technology

It realizes the reduction of the difference between the output standard text and the input text, improves the accuracy of file processing, reduces the need for manual adjustment, and improves efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114638226B_ABST
    Figure CN114638226B_ABST
Patent Text Reader

Abstract

The present application provides a text processing method, device, computer equipment and computer-readable storage medium, which belongs to the field of computer technology. The method comprises: obtaining at least one word segmentation result of an input text, each of the word segmentation results includes at least one type of phrase in the input text; obtaining at least one text to be processed in a standard text database according to each of the word segmentation results, the standard text database is used to store multiple standardized texts, each standardized text is used to identify address information; respectively determining the difference value between the input text and each of the texts to be processed; outputting a standard text matching the input text according to each of the difference values. The present application can reduce the difference between the output standard text and the input text, thereby achieving the effect of accuracy in file processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text processing method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] With the development of science and technology, computer equipment has entered people's work and life. People can store various information or data in computer equipment and conduct corresponding management, analysis and decision-making to improve efficiency.

[0003] In the related technology, for example, in the medical field, the patient's address and case waiting to be stored text can be input into a computer device, and these texts to be stored can be segmented to obtain at least one segmentation result for each text to be stored. Then, each text to be stored can be classified and stored, structured and stored according to each segmentation result of each text to be stored, or a standard text that meets specific rules can be output according to each text to be stored.

[0004] However, since the text to be stored may contain special symbols and typos, and the information recorded in the text to be stored may not be detailed, this solution has the problem that the output standard text is quite different from the text to be stored and the accuracy of the output standard text is low. Summary of the invention

[0005] The purpose of the present application is to provide a text processing method, apparatus, computer device and computer-readable storage medium, which can reduce the difference between the output standard text and the input text, thereby achieving the effect of accuracy in file processing.

[0006] The embodiment of the present application is implemented as follows:

[0007] According to a first aspect of an embodiment of the present application, a text processing method is provided, comprising:

[0008] Obtaining at least one word segmentation result of the input text, each of the word segmentation results including at least one type of phrase in the input text;

[0009] Acquire at least one text to be processed in a standard text database according to each of the word segmentation results, wherein the standard text database is used to store a plurality of standardized texts, each of which is used to identify address information;

[0010] respectively determining the difference values ​​between the input text and each of the to-be-processed texts;

[0011] According to each of the difference values, a standard text matching the input text is output.

[0012] Optionally, acquiring at least one to-be-processed text in a standard text database according to each of the word segmentation results includes:

[0013] Traversing the standard text database according to each of the word segmentation results, obtaining at least one standard phrase matching each of the word segmentation results, wherein the standard phrase is a phrase obtained by segmenting each of the standardized texts;

[0014] The at least one text to be processed is generated according to each of the standard phrases.

[0015] Optionally, acquiring at least one to-be-processed text in a standard text database according to each of the word segmentation results includes:

[0016] According to each of the word segmentation results and a preset order, sequentially acquiring at least one first text matching each of the word segmentation results from the standard text database;

[0017] Obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order;

[0018] Performing a completion process on each of the word segmentation results to obtain a completed word segmentation result, and acquiring at least one third text matching the updated word segmentation result from the standard text database in sequence according to the completed word segmentation result and the preset order;

[0019] The to-be-processed texts are determined according to the first texts, the second texts and the third texts.

[0020] Optionally, obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order, includes:

[0021] Obtaining a changed address corresponding to the address indicated by each segmentation result from a preset address change database, and updating the segmentation result based on the changed address to obtain an updated segmentation result;

[0022] According to the updated word segmentation result and the preset order, at least one second text matching the updated word segmentation result is acquired from the standard text database in sequence.

[0023] Optionally, determining each of the to-be-processed texts according to each of the first texts, each of the second texts, and each of the third texts includes:

[0024] Generate a text library to be processed according to the first text, the second text and the third text;

[0025] Determine whether there is duplicate text in the text library to be processed;

[0026] If so, other texts in the to-be-processed text library except the repeated texts are used as the to-be-processed texts;

[0027] If it does not exist, all the texts in the to-be-processed text library are used as the to-be-processed texts.

[0028] Optionally, the respectively determining the difference value between the input text and each of the to-be-processed texts includes:

[0029] Determining the minimum number of single-character edits required to convert the input text into the text to be processed;

[0030] Determining the number of identical phrases between the input text and each of the to-be-processed texts;

[0031] The difference value is calculated based on the minimum number of single-character edits and the number of identical phrases.

[0032] Optionally, the calculating the difference value according to the minimum number of single-character edits and the number of identical phrases includes:

[0033] Calculating the minimum edit distance between the input text and each of the to-be-processed texts according to the minimum number of single-character edits required to convert the input text into the to-be-processed texts;

[0034] Calculating the character distance between the input text and each of the to-be-processed texts according to the number of identical phrases existing between the input text and each of the to-be-processed texts;

[0035] The difference value is calculated according to the character distance and the minimum edit distance.

[0036] According to a second aspect of the embodiments of the present application, a text processing device is provided, the device comprising:

[0037] A first acquisition module, used to acquire at least one word segmentation result of an input text;

[0038] A second acquisition module, used for acquiring at least one text to be processed in a standard text database according to each of the word segmentation results;

[0039] A determination module, used for respectively determining the difference value between the input text and each of the to-be-processed texts;

[0040] An output module is used to output a standard text matching the input text according to each of the difference values.

[0041] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the text processing method described in the first aspect is implemented.

[0042] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text processing method described in the first aspect is implemented.

[0043] The beneficial effects of the embodiments of the present application include:

[0044] A text processing method provided by an embodiment of the present application obtains at least one word segmentation result of an input text, obtains at least one text to be processed in a standard text database according to each word segmentation result, determines the difference value between the input text and each text to be processed respectively, and outputs a standard text matching the input text according to each difference value. Among them, obtaining at least one word segmentation result of the input text can convert the input text into structured word segmentation results. Obtaining at least one text to be processed in a standard text database according to each word segmentation result can obtain a text to be processed that is the same as or relatively similar to the input text in the standard text database. Determining the difference value between the input text and each text to be processed respectively can determine the similarity or approximation between the input text and the text to be processed. According to each difference value, outputting a standard text matching the input text can ensure that the standard text matching the input text output according to the difference value has a high similarity or approximation with the input text. In this way, the difference between the output standard text and the input text can be reduced, thereby achieving the effect of accuracy of file processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of a first text processing method provided in an embodiment of the present application;

[0047] Figure 2 A flowchart of a second text processing method provided in an embodiment of the present application;

[0048] Figure 3A flowchart of a third text processing method provided in an embodiment of the present application;

[0049] Figure 4 A flowchart of a fourth text processing method provided in an embodiment of the present application;

[0050] Figure 5 A flowchart of a fifth text processing method provided in an embodiment of the present application;

[0051] Figure 6 A flowchart of a sixth text processing method provided in an embodiment of the present application;

[0052] Figure 7 A flowchart of a seventh text processing method provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of the structure of a text processing device provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0057] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0058] In the medical field, in order to reduce the workload of relevant staff, the patient's address and case waiting to be stored text can generally be input into a computer device, and these texts to be stored can be segmented to obtain at least one segmentation result of each text to be stored, and then each text to be stored can be classified and stored according to each segmentation result of each text to be stored, or a standard text that meets specific rules can be output according to each text to be stored. However, since the text to be stored may contain special symbols and typos, and the information recorded in the text to be stored may not be detailed, this solution has the problem that the output standard text is quite different from the text to be stored, and the accuracy of the output standard text is low.

[0059] In addition, generally speaking, if there is a large difference between the output standard text and the text to be stored, relevant staff will need to manually adjust the output standard text, which will also lead to the problem of low efficiency.

[0060] To this end, an embodiment of the present application provides a text processing method, which obtains at least one word segmentation result of an input text, obtains at least one text to be processed in a standard text database based on each word segmentation result, and determines the difference values ​​between the input text and each text to be processed respectively. Based on each difference value, a standard text matching the input text is output, which can reduce the difference between the output standard text and the input text, thereby achieving the effect of accuracy in file processing.

[0061] The embodiment of the present application is described by taking the text processing method for address information applied in a computer device as an example, but it does not mean that the embodiment of the present application can only be applied to text processing for address information.

[0062] The computer device may be a computer, a tablet computer, a smart phone, or any other electronic device with a processing function, which is not limited in the present embodiment of the application.

[0063] The text processing method provided in the embodiment of the present application is explained in detail below.

[0064] Figure 1 This is a flowchart of a text processing method provided by the present application. The method can be applied to a computer device, which can be the aforementioned terminal device or server. Figure 1 , the present application embodiment provides a text processing method, including:

[0065] Step 1001: Obtain at least one word segmentation result of the input text.

[0066] Optionally, the input text may be a text including address information, or a text including case information, or a text including any possible information, which is not limited in the embodiment of the present application.

[0067] The user or related technicians can input the input text by touching the screen of the computer device, or by using a peripheral device connected to the computer device such as a mouse, keyboard, electronic stylus, or electronic handwriting tablet, etc. The present application embodiment does not limit this.

[0068] Optionally, the input text may be segmented using a string-based segmentation algorithm, a comprehension-based segmentation algorithm, a statistics-based segmentation algorithm, or any other possible segmentation algorithm to obtain segmentation results.

[0069] Specifically, the Jieba word segmentation algorithm may be used to segment the input text to obtain each word segmentation result. This embodiment of the present application does not limit this.

[0070] In addition, word segmentation may refer to the operation of dividing a Chinese character sequence into multiple separate words. This Chinese character sequence may be the input text mentioned above.

[0071] Optionally, each word segmentation result includes at least one type of phrase in the input text.

[0072] Optionally, each type may include an address, a person's name, an administrative division name, a disease name, a drug name, etc.

[0073] For example, if the input text includes address information, the Jieba word segmentation algorithm can be used to segment the address phrases in the input text according to multi-level administrative divisions.

[0074] For example, the administrative region can be divided into four levels of administrative divisions, specifically, the first level provincial administrative region: province, autonomous region, municipality directly under the central government, special administrative region, etc., the second level prefecture-level administrative region: prefecture-level city, autonomous prefecture, league, etc., the third level county-level administrative region: city district, county, autonomous county, county-level city, banner, etc., the fourth level township-level administrative region: street office, town, township, etc. This embodiment of the application is not limited to this.

[0075] For another example, the administrative region can be divided into three levels of administrative divisions, specifically, a first-level provincial administrative region, a second-level prefectural administrative region, and a third-level county administrative region. The administrative region can also be divided into five levels of administrative divisions, specifically, a first-level provincial administrative region, a second-level prefectural administrative region, a third-level county administrative region, a fourth-level township administrative region, and a fifth-level village administrative region. This embodiment of the application is not limited to this.

[0076] For example, assuming that the input text is "615 Hutai Branch Road, Pengpu Town, Jing'an District, Shanghai", and the Jieba word segmentation algorithm is used to segment multiple address phrases according to the above four-level administrative divisions, then the address phrases can be "Shanghai", "Jing'an District" and "Pengpu Town". Since "615 Hutai Branch Road" does not belong to the above four-level administrative divisions, "615 Hutai Branch Road" will not be segmented as a word segmentation result.

[0077] If multiple address phrases are segmented according to the above three-level administrative divisions, then each address phrase can be "Shanghai" and "Jing'an District" respectively. If multiple address phrases are segmented according to the above five-level administrative divisions, then each address phrase can be "Shanghai", "Jing'an District", "Pengpu Town" and "No. 615 Hutai Branch Road" respectively. The specific method of segmenting the input text can be selected according to actual needs. This embodiment of the application does not limit this.

[0078] It is worth noting that by obtaining at least one word segmentation result of the input text, the input text entered by the user or relevant technical personnel can be converted into structured word segmentation results, which is convenient for subsequent text processing based on the word segmentation results. In addition, by using the word segmentation algorithm for word segmentation, the word segmentation result of the input text can be obtained accurately and efficiently, so that the accuracy and efficiency of text processing can be improved.

[0079] Step 1002: Obtain at least one to-be-processed text in a standard text database according to each word segmentation result.

[0080] Optionally, the standard text database is used to store multiple standardized texts, and each standardized text is used to identify address information.

[0081] Each standardized text may be a text based on a standard administrative division, and each standardized text may include a provincial administrative region, a prefectural administrative region, a county administrative region, and / or a township administrative region. For example, a standardized text may be Xinqiao Street, Yi'an District, Tongling City, Anhui Province, where Anhui Province is a standard provincial administrative region, Tongling City is a standard prefectural administrative region, Yi'an District is a standard county administrative region, and Xinqiao Street is a standard township administrative region.

[0082] In one possible manner, each standardized text stored in the standard text database may be segmented according to a preset segmentation condition to obtain each standard phrase matching the standardized text.

[0083] Optionally, the preset word segmentation condition may refer to using the same word segmentation algorithm as in step 1001 to segment each standard phrase including address information in each standardized text according to multi-level administrative divisions.

[0084] For example, a standardized text may be Xinqiao Street, Yi'an District, Tongling City, Anhui Province. Then, the standard phrases matching this standardized text are "Anhui Province", "Tongling City", "Yi'an District" and "Xinqiao Street" respectively.

[0085] For example, a standardized text may be Donghuamen Street, Dongcheng District, Beijing. Then, the standard phrases matching this standardized text are "Beijing City", "district", "Dongcheng District", and "Donghuamen Street" respectively.

[0086] See Table 1, in which FullText is the full text of the standardized text, Id is the field code corresponding to FullText, adminDivId is the administrator code corresponding to FullText, adminDivId is the management type code corresponding to FullText, parentAdminDivId is the parent administrator code, provinceCode is the provincial administrative region code, provinceName is the provincial administrative region name, cityCode is the prefecture-level administrative region code, cityName is the prefecture-level administrative region name, countyCode is the county-level administrative region code, countyName is the county-level administrative region name, townCode is the township-level administrative region code, and townName is the township-level administrative region name.

[0087] As shown in Example 1 in Table 1, the full text of the standardized text is Donghuamen Street, Dongcheng District, Beijing, and the ID is 3670. As shown in Example 2 in Table 1, the full text of the standardized text is Xinqiao Street, Yi'an District, Tongling City, Anhui Province, and the ID is 15489.

[0088] Table 1

[0089]

[0090]

[0091] Furthermore, each standardized text stored in the standard text database may have a corresponding ID, so that each standardized text can be quickly, accurately and conveniently found according to the ID of each standardized text.

[0092] Optionally, all prefecture-level administrative regions under the same provincial administrative region have an index relationship with the provincial administrative region, all county-level administrative regions under the same prefecture-level administrative region have an index relationship with the prefecture-level administrative region, and all township-level administrative regions under the same county-level administrative region have an index relationship with the county-level administrative region. This embodiment of the application is not limited to this. In addition, if the standard phrases stored in the standard text database are phrases segmented from the same standardized text, then there is a corresponding index relationship between the standard phrases.

[0093] The standardized text of "Donghuamen Street, Dongcheng District, Beijing" stored in the standard text database is used as an example for explanation. See Table 2, which lists five standardized texts and the IDs corresponding to each standardized text.

[0094] It can be seen that the ID corresponding to "Dongcheng District, Beijing" is 1, the ID corresponding to "Donghuamen Street, Dongcheng District, Beijing" is 3670, the ID corresponding to "Jingshan Street, Dongcheng District, Beijing" is 3671, the ID corresponding to "Jiaodaokou Street, Dongcheng District, Beijing" is 3672, and the ID corresponding to "Andingmen Street, Dongcheng District, Beijing" is 3673.

[0095] Then, from Table 2, we can get the key-value pair of Dongcheng District in Beijing as {'Dongcheng District':[1,3670,3671,3672,3673,…]}. In this way, we can quickly, conveniently and accurately find the township-level administrative districts that have an index relationship with Dongcheng District.

[0096] Table 2

[0097]

[0098]

[0099] In this way, each standard text stored in the standard text database can be segmented into multiple standard phrases, and the address information identified by each standard text can be segmented and processed separately by provincial, prefectural, county, and township levels, so as to facilitate matching the corresponding standard phrases according to each segmentation result.

[0100] Optionally, the operation of obtaining at least one text to be processed in the standard text database according to each word segmentation result may be to search the standard text database for standard phrases that are identical or similar to each word segmentation result, and combine these standard phrases that are identical or similar to each word segmentation result, and use the text obtained after the combination as the text to be processed. This embodiment of the application is not limited to this.

[0101] It is worth noting that, in this way, the text to be processed that matches each word segmentation result in the standard text database can be obtained, and each text to be processed is obtained by combining standard phrases based on the standardized text segmentation of the standard administrative division. In this way, the text to be processed that is the same or relatively similar to the input text in the standard text database can be obtained, which is convenient for subsequent corresponding operations based on each text to be processed, and the difference between the output text and the input text can be reduced, thereby achieving the effect of accuracy in file processing.

[0102] Step 1003: respectively determine the difference values ​​between the input text and each to-be-processed text.

[0103] Optionally, the difference value may be used to characterize the similarity or approximation between the input text and the text to be processed.

[0104] Generally, the smaller the difference value, the more similar the input text and the text to be processed are, the larger the difference value, the less similar the input text and the text to be processed are, and if the difference value is 0, the input text and the text to be processed are the same. This embodiment of the application does not limit this.

[0105] Optionally, the difference value may be determined by a semantic recognition algorithm or any other possible method, which is not limited in the present embodiment of the application.

[0106] It is worth noting that by calculating the difference value, the similarity or approximation between the input text and the text to be processed can be determined, which facilitates subsequent operations.

[0107] Step 1004: Output the standard text matching the input text according to each difference value.

[0108] Optionally, the one to be processed text with the smallest difference value from the input text among the to be processed texts can be used as the output standard text that matches the input text. The two to be processed texts with the smallest difference value from the input text among the to be processed texts can also be used as the output standard text that matches the input text. The specific way of outputting the standard text can be set according to actual needs. The embodiments of the present application do not limit this.

[0109] Optionally, the standard text matching the input text may be a text to be processed that has the smallest difference value with the input text among the texts to be processed. And since each text to be processed is obtained by segmenting each standard text stored in the standard text database into a plurality of standard phrase combinations, each text to be processed is also a standardized text.

[0110] It is worth noting that since the smaller the difference value, the higher the similarity or approximation between the input text and the text to be processed corresponding to the difference value, it can be ensured that the standard text output according to the difference value that matches the input text has a high degree of similarity or approximation to the input text, thereby reducing the difference between the output standard text and the input text, thereby achieving the effect of accuracy in file processing.

[0111] In an embodiment of the present application, by obtaining at least one word segmentation result of the input text, obtaining at least one to-be-processed text in the standard text database according to each word segmentation result, respectively determining the difference value between the input text and each to-be-processed text, and outputting the standard text matching the input text according to each difference value. Among them, obtaining at least one word segmentation result of the input text can convert the input text into structured word segmentation results. Obtaining at least one to-be-processed text in the standard text database according to each word segmentation result can obtain the to-be-processed text in the standard text database that is the same as or relatively similar to the input text. Determining the difference value between the input text and each to-be-processed text respectively can determine the similarity or approximation between the input text and the to-be-processed text. According to each difference value, outputting the standard text matching the input text can ensure that the standard text matching the input text output according to the difference value has a high similarity or approximation with the input text. In this way, the difference between the output standard text and the input text can be reduced, thereby achieving the effect of accuracy of file processing.

[0112] In a possible implementation, in order to improve the accuracy of each word segmentation result, see Figure 2 , obtain at least one word segmentation result of the input text, including:

[0113] Step 1005: extract at least one sub-information of the input text based on a preset word segmentation condition.

[0114] Optionally, the input text includes various sub-information, and each sub-information is different.

[0115] Optionally, the preset word segmentation condition may refer to using the Jieba word segmentation algorithm to segment the input text into phrases including address information in the input text according to multi-level administrative divisions.

[0116] Alternatively, the sub-information may be a phrase including address information.

[0117] For example, if the input text is "The mayor of Pengpu Town, Jing'an District, Shanghai", and the Jieba word segmentation algorithm is used to segment multiple sub-information according to the above four-level administrative divisions, then the sub-information can be "Shanghai", "Jing'an District" and "Pengpu Town". Since "mayor" is not a phrase that includes address information and does not belong to the above four-level administrative divisions, "mayor" will not be segmented as sub-information.

[0118] Step 1006: Use each sub-information as a word segmentation result.

[0119] In this way, each word segmentation result including the address information corresponding to the input text can be accurately and efficiently obtained, thereby improving the accuracy and efficiency of text processing.

[0120] In a possible implementation, in order to improve the accuracy of the obtained text to be processed, see Figure 3 , obtaining at least one to-be-processed text in the standard text database according to each word segmentation result, including:

[0121] Step 1007: traverse the standard text database according to each word segmentation result to obtain at least one standard phrase matching each word segmentation result.

[0122] Optionally, the at least one standard phrase matching each word segmentation result may refer to a standard phrase stored in the standard text database that is the same as or similar to each word segmentation result.

[0123] Optionally, the standard phrase may be a phrase obtained by segmenting each standardized text using the above-mentioned word segmentation algorithm.

[0124] Step 1008: Generate the at least one text to be processed according to each standard phrase.

[0125] Furthermore, the operation of generating the at least one text to be processed according to each standard phrase may be to arrange and combine each standard phrase according to the size of the administrative division, and use the text obtained by combining each standard phrase as the text to be processed.

[0126] For example, if four standard phrases matching each word segmentation result are obtained, and these four standard phrases are "Henan Province", "Zhengzhou City", "Xinzheng City", and "Xinmi City", then these four standard phrases can be arranged and combined in the order of provincial administrative regions, prefecture-level administrative regions, and county-level administrative regions. Since "Xinzheng City" and "Xinmi City" are both county-level administrative regions, two texts can be obtained after combining these four standard phrases, and these two texts are "Xinzheng City, Zhengzhou City, Henan Province" and "Xinmi City, Zhengzhou City, Henan Province". Therefore, two texts to be processed can be generated according to each standard phrase, namely "Xinzheng City, Zhengzhou City, Henan Province" and "Xinmi City, Zhengzhou City, Henan Province".

[0127] In this way, accurate texts to be processed can be obtained, and the difference between the output standard text and the input text can be reduced, thereby achieving the effect of accuracy in file processing.

[0128] In a possible implementation, in order to improve the accuracy and comprehensiveness of each text to be processed, see Figure 4 , obtaining at least one to-be-processed text in the standard text database according to each word segmentation result, including:

[0129] Step 1009: According to each word segmentation result and a preset order, at least one first text matching each word segmentation result is obtained from the standard text database in sequence.

[0130] Optionally, the first text may be a standardized text stored in the standard text database.

[0131] Specifically, each standard phrase in the standard text database that matches, is similar to, or is identical to each segmentation result can be determined according to each segmentation result, and each standard phrase is arranged and combined according to the level of the administrative division, and the text obtained after the arrangement and combination is used as each first text. This embodiment of the application is not limited to this.

[0132] Optionally, the preset order may refer to an order from small to large administrative divisions, which is not limited in the present embodiment of the application.

[0133] For example, if the administrative region is divided into four levels of administrative divisions, and the four levels of administrative divisions are provincial-level administrative regions, prefectural-level administrative regions, county-level administrative regions and / or township-level administrative regions, then the preset order can be the order of township-level administrative regions, county-level administrative regions, prefectural-level administrative regions, and provincial-level administrative regions.

[0134] For another example, if the administrative region is divided into three levels of administrative divisions, and the three levels of administrative divisions are provincial-level administrative regions, prefecture-level administrative regions, and county-level administrative regions, then the preset order can be the order of county-level administrative regions, prefecture-level administrative regions, and provincial-level administrative regions.

[0135] In addition, when obtaining each first text from the standard text database in sequence according to the preset order, if a text matching the word segmentation result can be obtained based on the word segmentation result of any level of administrative division, then the corresponding standardized texts or standard phrases including the address information of the upper level administrative division can be indexed according to the text matching the word segmentation result. And, the text matching the word segmentation result and the standardized texts or standard phrases indexed by the text matching the word segmentation result are used as the first texts. This embodiment of the present application does not limit this.

[0136] It is worth noting that since there is an index relationship between the standardized texts stored in the standard text database, and the ID of the next-level administrative district can be indexed through the ID of the previous-level administrative district, or the ID of the previous-level administrative district can be indexed through the ID of the next-level administrative district, the ID of the administrative district at each level in the standard text database can be obtained.

[0137] That is to say, when the lowest level administrative division is used to obtain each first text from the standard text database, the first text of each administrative division that has an index relationship with the Id of the lowest level administrative division can be obtained based on the Id of the lowest level administrative division.

[0138] For example, the administrative division can be divided into three levels of administrative divisions, and the three levels of administrative divisions are provincial administrative divisions, prefectural administrative divisions, and county administrative divisions. If the ID of a county administrative division is used for indexing, the ID of the prefectural administrative division that has an index relationship with the county administrative division and the ID of the provincial administrative division that has an index relationship with the county administrative division can be indexed, and then the first text of the prefectural administrative division that has an index relationship with the county administrative division and the first text of the provincial administrative division that has an index relationship with the county administrative division can be obtained.

[0139] If the ID of a prefecture-level administrative region is used for indexing, the ID of the provincial administrative region that has an index relationship with the prefecture-level administrative region can be indexed, and the IDs of all county-level administrative regions that have an index relationship with the prefecture-level administrative region can also be indexed, and then the first text of the provincial administrative region that has an index relationship with the prefecture-level administrative region and the first text of all county-level administrative regions that have an index relationship with the prefecture-level administrative region can be obtained. This embodiment of the application is not limited to this.

[0140] Step 1010: obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order.

[0141] It is worth noting that, for example, if the input text entered by the user or relevant technical personnel is "Tongguanshan District, Tongling City, Anhui Province" or "Shizishan District, Tongling City, Anhui Province", and the Tongguanshan District and Shizishan District of Tongling City have been abolished and the Tongguan District of Tongling City has been established in the original jurisdiction of the Tongguanshan District and Shizishan District of Tongling City. If "Tongguanshan District, Tongling City, Anhui Province" or "Shizishan District, Tongling City, Anhui Province" is directly segmented, then the text to be processed in the standard text database obtained based on the segmentation results is inaccurate. Therefore, in order to avoid this situation and improve the accuracy of the acquired text to be processed, the input text needs to be updated.

[0142] For another example, if the input text is "Tongling County, Tongling City, Anhui Province", then the word segmentation results can be "Anhui Province", "Tongling City", and "Tongling County" respectively. Since Tongling County, Tongling City has been abolished and Yi'an District of Tongling City has been established, the changed address corresponding to the address indicated by each word segmentation result is Yi'an District, Tongling City, Anhui Province. The updated word segmentation results can be "Anhui Province", "Tongling City", and "Yi'an District" respectively.

[0143] In this way, the problem of inaccurate matching of the to-be-processed text caused by changes in administrative area or name of the address indicated by the input text entered by the user or relevant technical personnel or the address indicated by each word segmentation result of the input text can be avoided.

[0144] Step 1011: completing each word segmentation result to obtain a completed word segmentation result, and obtaining at least one third text matching the updated word segmentation result from the standard text database in sequence according to the completed word segmentation result and the preset order.

[0145] It is worth noting that, for example, if the input text entered by the user or relevant technical personnel is "Aksu, Xinjiang", and the standardized text of the Xinjiang Uygur Autonomous Region indicated by the storage in the standard text database is "Xinjiang Uygur Autonomous Region", the standardized text of the Aksu region indicated by the storage in the standard text database is "Aksu Region". If the word segmentation result for "Aksu, Xinjiang" is directly used, the text to be processed in the standard text database obtained according to the obtained word segmentation result is inaccurate. Therefore, in order to avoid this situation and improve the accuracy of the obtained text to be processed, it is necessary to complete the input text, and then obtain each third text matching the updated word segmentation result from the standard text database according to the completed word segmentation result.

[0146] For example, if the input text is "Aksu, Xinjiang", then the word segmentation results can be "Xinjiang" and "Aksu" respectively. Then, "Xinjiang" and "Aksu" need to be completed as "Xinjiang Uygur Autonomous Region" and "Aksu Region" respectively. Then, according to the above preset order, specifically in the order of administrative divisions from small to large, the third texts matching the word segmentation results can be obtained from the standard text database in sequence.

[0147] Specifically, at least one text matching "Aksu Region" can be first obtained from the standard text database based on "Aksu Region". If at least one text can be matched, other texts that have an index relationship with each text can be directly queried based on each text matching "Aksu Region", and each text matching "Aksu Region" and other texts indicating the upper-level administrative division that have an index relationship with each text can be used as each third text.

[0148] If no text is matched in the standard text database according to "Aksu Region", it is necessary to obtain at least one text matching "Xinjiang Uygur Autonomous Region" from the standard text database according to "Xinjiang Uygur Autonomous Region".

[0149] Optionally, each word segmentation result may be completed and each third text may be matched by utilizing character co-occurrence.

[0150] Optionally, each third text may be matched by the following formula (1):

[0151]

[0152] Among them, s1 is the completed word segmentation result corresponding to each word segmentation result, s2 is the text matched based on the completed word segmentation result, |s1| is the length of the completed word segmentation result, |s2| is the length of the text matched based on the completed word segmentation result, m is the number of identical characters in the text matched based on the completed word segmentation result and the completed word segmentation result, s is the character co-occurrence score, the higher the character co-occurrence score, the more similar the completed word segmentation result and the text matched based on the completed word segmentation result are.

[0153] Optionally, the first x texts with the highest character collinearity scores matched based on the word segmentation results after completion are used as the third texts. Generally, x can be set to 20, and of course x can also be set to other values. This embodiment of the application does not limit this.

[0154] Step 1012: Determine each text to be processed according to each first text, each second text, and each third text.

[0155] It is worth noting that by determining each first text, each second text and each third text respectively in a variety of different ways, it can be ensured that the standardized text or standard phrase corresponding to each word segmentation result of the input text can be determined to the greatest extent. Then, each text to be processed is determined based on each first text, each second text and each third text, so that the comprehensiveness and accuracy of determining each text to be processed can be greatly improved. Then, according to each text to be processed, the subsequent difference value calculation is performed and the standard text is output according to the difference value, so as to reduce the difference between the output standard text and the input text, thereby achieving the effect of accuracy of file processing.

[0156] For a possible implementation, see Figure 5 , obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order, including:

[0157] Step 1013: Obtain the changed address corresponding to the address indicated by each segmentation result from a preset address change database, and update the segmentation result based on the changed address to obtain an updated segmentation result.

[0158] Optionally, the updated word segmentation result may refer to the latest and most accurate administrative division information corresponding to the address indicated by each word segmentation result. For example, a word segmentation result is "Chang'an County", and the address indicated is Chang'an County, Xi'an City, Shaanxi Province, and Chang'an County has been abolished as a county and established as a district, then the updated word segmentation result corresponding to this word segmentation result may be "Chang'an District". This embodiment of the application does not limit this.

[0159] Optionally, the preset address change database may be a database established in advance based on the standard text database.

[0160] Optionally, the preset address change database may store change data of each standardized text.

[0161] Exemplarily, if each standardized text is address information, then the preset address change database can store the historical place names corresponding to each standardized text.

[0162] For example, Tongling City's Tongguanshan District and Shizishan District have both been abolished and Tongling City's Tongguan District has been established in the original jurisdictions of Tongling City's Tongguanshan District and Shizishan District. Then, the historical name change data corresponding to Tongling City's Tongguan District can be stored in the preset address change database, such as {"Tongguan District, Tongling City, Anhui Province":["Tongguanshan District, Tongling City, Anhui Province","Tongguan District, Tongling City, Anhui Province"]}.

[0163] For another example, Tongling County of Tongling City has been abolished and Yian District of Tongling City has been established. Then, the historical name change data corresponding to Yian District of Tongling City can be stored in the preset address change database, for example, {"Yian District of Tongling City, Anhui Province":["Tongling County of Tongling City, Anhui Province"]}.

[0164] In this way, the address information in the preset address change database can be matched with the standardized text stored in the standard text database, and the historical name change data of the address indicated by each word segmentation result can be determined, and the changed address corresponding to the address indicated by each word segmentation result can be determined, thereby obtaining the updated word segmentation result.

[0165] Step 1014: According to the updated word segmentation result and the preset order, at least one second text matching the updated word segmentation result is obtained from the standard text database in sequence.

[0166] It is worth noting that in this way, based on the latest and most accurate administrative division information corresponding to the address indicated by each segmentation result, each second text matching the updated segmentation result can be obtained in the standard text database, and the second text matching each updated segmentation result can also be obtained from the standard text database in sequence according to the above-mentioned preset order, specifically in order of administrative divisions from small to large.

[0167] In this way, the accuracy of determining each second text can be greatly improved, and thus the accuracy of determining each text to be processed can be improved.

[0168] For a possible implementation, see Figure 6 , respectively determine the difference values ​​between the input text and each to-be-processed text, including:

[0169] Step 1015: Determine the minimum number of single-character edits required to convert the input text into the text to be processed.

[0170] Optionally, the minimum number of single-character edits can be used to characterize the degree of difference between any two strings. The logic for calculating the minimum number of single-character edits is to determine how many processing steps are required at least to transform one string into another string. Generally, when calculating the minimum number of single-character edits, only one character can be inserted, replaced, or deleted each time the string is edited.

[0171] Optionally, the minimum number of single-character edits can be calculated by the Minimum Edit Distance (MED) algorithm.

[0172] Exemplarily, assume that the input text is "Suqiao Township, Shangrao County, Jiangxi Province", and the text to be processed is "Suqiao Township, Wannian County, Shangrao City, Jiangxi Province". Then, calculating the minimum number of single-character edits between the input text and the text to be processed is to calculate the number of edits required to change "Suqiao Township, Shangrao County, Jiangxi Province" into "Suqiao Township, Wannian County, Shangrao City, Jiangxi Province".

[0173] The specific editing process can be as follows:

[0174] (1) First, replace the "County" in "Suqiao Township, Shangrao County, Jiangxi Province" with "City", then it can be changed to "Suqiao Township, Shangrao City, Jiangxi Province".

[0175] (2) Insert "Wan" into "Suqiao Township, Shangrao City, Jiangxi Province", then it can be changed to "Suqiao Township, Shangrao City, Wan Jiangxi Province".

[0176] (3) Insert "Nian" into "Suqiao Township, Shangrao City, Wan Jiangxi Province", then it can be changed to "Suqiao Township, Shangrao City, Wannian Jiangxi Province".

[0177] (4) Insert "County" into "Suqiao Township, Shangrao City, Wannian Jiangxi Province", then it can be changed to "Suqiao Township, Wannian County, Shangrao City, Jiangxi Province".

[0178] At this time, the editing process is completed.

[0179] As can be seen from the above editing process, the edit distance between "Suqiao Township, Shangrao County, Jiangxi Province" and "Suqiao Township, Wannian County, Shangrao City, Jiangxi Province" is 4, and the minimum number of single-character edits is 4.

[0180] Step 1016: Determine the number of identical phrases existing between the input text and each text to be processed.

[0181] Optionally, the number of identical phrases between the input text and each to-be-processed text can be determined by an N-Gram distance algorithm. The number of identical phrases between the input text and each to-be-processed text can also be determined by the Longest Common Subsequence (LCS), cosine similarity, and levenshtein functions. Of course, the number of identical phrases between the input text and each to-be-processed text can also be determined by any other possible algorithm. This embodiment of the present application does not limit this.

[0182] For example, taking the N-Gram distance algorithm as an example, the N-Gram distance algorithm is an algorithm based on a statistical language model. The N-Gram distance algorithm can be used to divide the input text into multiple segments of length N, and each segment can be recorded as a Gram. If N=2, it is a 2-gram distance algorithm, if N=3, it is a 3-gram distance algorithm, and so on. In the embodiment of the present application, the 2-Gram distance algorithm is used as an example for explanation, but it does not mean that in the embodiment of the present application, only the 2-Gram distance algorithm can be used to determine the number of identical phrases between the input text and each text to be processed.

[0183] For example, if the input text is "No. 200 Renmin Avenue, Huangpu District, Shanghai", the 2-Gram distance algorithm can be used to segment the input text into the segments "Shanghai", "Hai Shi", "Shi Huang", "Huangpu", "Pu District", "Qu Ren", "Renmin", "Min Da", "Avenue", "Avenue 2", "20", "00", and "No. 0".

[0184] For another example, if the input text is "Xingfu Road, Xin'an Town, Xinyi City, Jiangsu Province", and one of the texts to be processed is "Xin'an Sub-district, Xinyi City, Xuzhou City, Jiangsu Province", then the 2-Gram distance algorithm can be used to segment the input text and this text to be processed respectively. The segments of the input text can be obtained as {('江 ','苏'):1,('苏','省'):1,('省','新'):1,('新','沂'):1,('沂','市'):1,('市','新'):1,('新','安'):1,(' 安','镇'):1,('镇','幸'):1,('幸','福'):1,('福','路'):1}, and the segments of this text to be processed are {('市',' 新'):2,('江','苏'):1,('苏','省'):1,('省','徐'):1,('徐','州'):1,('州','市'):1,('新','沂'):1,('沂 ','市'):1,('新','安'):1,('安','街'):1,('街','道'):1}. The numbers therein are the repetition times of each segment.

[0185] The identical phrases existing between the input text and this text to be processed can be determined from the segments of the input text and the segments of this text to be processed as {('江','苏'),('苏','省'),('新','沂'),('沂','市')(' 市','新'),('新','安')}. Then, the number of identical phrases existing between the input text and each text to be processed is 6.

[0186] Step 1017: Calculate the difference value according to the minimum single-character editing times and the number of identical phrases.

[0187] It should be noted that since the minimum single-character editing times can be used to represent the difference degree between any two strings, and the number of identical phrases is the number of identical phrases existing between the input text and each text to be processed, then calculating the difference value between the input text and each text to be processed according to the minimum single-character editing times and the number of identical phrases can accurately reflect the similarity degree or approximation degree between the input text and each text to be processed. In this way, it is possible to reduce the difference between the output standard text and the input text, and further achieve the effect of the accuracy of file processing.

[0188] In a possible implementation manner, refer to Figure 7 , calculating the difference value according to the minimum single-character editing times and the number of identical phrases includes:

[0189] Step 1018: Calculate the minimum edit distance between the input text and each to-be-processed text according to the minimum number of single-character edits required to convert the input text into the to-be-processed text.

[0190] Optionally, the minimum number of single-character edits may be directly used as the minimum edit distance.

[0191] Optionally, the minimum edit distance can be calculated by the following formula (2):

[0192]

[0193] Among them, lev a,b (i, j) is the minimum edit distance, a and b represent the input text and the text to be processed respectively, i represents the first i characters in the input text, and j represents the first j characters in the text to be processed. When i=|a|, j=|b|, lev a,b The value of (i, j) is lev a,b (|a|, |b|), max(i, j) represents the largest value between i and j.

[0194] When min(i,j)=0, that is, when at least one of i and j is 0, it means that at least one of the input text a and the to-be-processed text b is an empty string. In this case, only max(i,j) single-character operations are required, such as single-character deletion or single-character insertion, to replace the first i characters a in the to-be-processed text. i Convert to the first j characters b in the text to be processed j .

[0195] When min(i,j)≠0, that is, when the values ​​of i and j are not 0, lev a,b (i, j) is the following three editing methods to make the first i characters a in the text to be processed i Convert to the first j characters b in the text to be processed j , and calculate the first i characters a in the text to be processed i Convert to the first j characters b in the text to be processed j The minimum edit distance is:

[0196] The first method is as follows: lev in formula (2) a,b (i-1,j)+1 indicates deleting the i-th character in the input text a. When deleting characters in the input text a, only one character can be deleted each time. In this case, lev a,b (i,j) is lev a,b(i-1,j)+1, each time a character is deleted, the edit distance between the input text a and the text to be processed b increases by 1.

[0197] The second method is as follows: lev in formula (2) a,b (i,j-1)+1 indicates inserting a character at the position of the i-th character in the input text a. i When inserting characters in , only one character can be inserted per edit. In this case, lev a,b (i,j) is lev a,b (i,j-1)+1, each time a character is inserted, the edit distance between the input text a and the text to be processed b increases by 1.

[0198] The third method is as follows: lev in formula (2) a,b (i-1,j-1)+I represents the replacement operation of the i-th character in the input text a.

[0199] Among them, when the replacement is completed, the first i characters a in the input text a i and the first j characters b in the text to be processed j The same, that is, when a i =b j When ai≠bj, I takes 0. When the replacement is completed, if the first i characters in the input text a and the first j characters bj in the text to be processed are not the same, that is, when ai≠bj, I takes 1.

[0200] In addition, replace the first i characters a in the input text a i When you replace characters in the , you can only replace one character at a time. In this case, lev a,b (i,j) is lev a,b (i-1,j-1)+I, each time a character is replaced, the edit distance between the input text a and the text to be processed b increases by 1.

[0201] That is, the first i characters a in the input text a can be edited by inserting, replacing, deleting, inserting and replacing, deleting and replacing, or inserting, replacing and deleting. i The editing method is to edit the first i characters a in the input text a i The resulting string is the same as the first j characters b in the text b to be processed j The present application embodiment does not limit this.

[0202] Furthermore, if multiple editing methods are required to edit the first i characters a in the text to be processed i Only the first i characters a in the text to be processed can be processedi Convert to the first j characters b in the text to be processed j , then, during the editing process, the order of performing the insertion operation, replacement operation, and deletion operation can be in any order.

[0203] For example, if it is necessary to perform the insertion operation, replacement operation, and deletion operation to convert the first i characters a in the text to be processed i Convert to the first j characters b in the text to be processed j , then, the insertion operation can be performed first, followed by the replacement operation, and finally the deletion operation. Or the replacement operation can be performed first, followed by the deletion operation, and finally the insertion operation. Or the deletion operation can be performed first, followed by the replacement operation, and finally the insertion operation. Naturally, it can also be edited in any other possible order. The embodiments of the present application do not limit this.

[0204] For another example, the first i characters a in the text to be processed can be converted to the first j characters b in the text to be processed through various editing methods i Convert to the first j characters b in the text to be processed j , and calculate the edit distance corresponding to each editing method for converting the first i characters a in the text to be processed i Convert to the first j characters b in the text to be processed j , and then take the smallest one of the calculated edit distances as the above-mentioned minimum edit distance. And the editing method corresponding to the minimum edit distance is the optimal editing method for converting the input text to the text to be processed.

[0205] Exemplarily, the input text a can be "two five six", and the text to be processed b can be "five nine nine eight four". Then, the first 3 characters of the input text a can be converted to the first 5 characters of the text to be processed b in the following way:

[0206] First step, determine i = 3, j = 5, and then insert the fifth character "four" of the text to be processed b after the third character of the input text a. The input text a becomes "two five six four".

[0207] Second step, determine i = 3, j = 4, and then insert the fourth character "eight" of the text to be processed b after the third character of the input text a. The input text a becomes "two five six eight four".

[0208] Third step, determine i = 3, j = 3, and then insert the third character "nine" of the text to be processed b after the third character of the input text a. The input text a becomes "two five six nine eight four".

[0209] Step 4: Determine that i = 3 and j = 2, and then replace the 3rd character of the input text a with the 2nd character "nine" of the text b to be processed. The input text a then becomes "two five nine nine eight four".

[0210] Since the 2nd character of the input text a and the 1st character of the text b to be processed are both "five", there is no need to process the 2nd character of the input text a.

[0211] Therefore, step 5 is to determine that i = 1 and j = 0, and then delete the 1st character of the input text a. The input text a then becomes "five nine nine eight four".

[0212] It can be seen that by this method, the edit distance calculated by converting the first 3 characters of the input text a to the first 5 characters of the text b to be processed is 5.

[0213] Through the above five steps, the first 3 characters of the input text a can be converted to the first 5 characters of the text b to be processed. Of course, the first 3 characters of the input text a can also be converted to the first 5 characters of the text b to be processed by other methods. In addition, the edit distances calculated by other possible editing methods are also greater than or equal to 5. Therefore, the minimum edit distance for converting "two five six" to "five nine nine eight four" can be obtained as 5.

[0214] The above five steps are only an example of a possible editing method given by the embodiments of the present application for explanation, and do not mean that the input text a can only be converted in the editing method listed in these five steps in this case. The embodiments of the present application do not limit this.

[0215] In summary, when min(i,j)≠0, the minimum edit distance for converting the first i characters ai of the text to be processed to the first j characters bj of the text to be processed is the minimum of the edit distances calculated by the above three methods or various combinations of these three methods. i The character distance is used to characterize the difference between the input text and each text to be processed. Generally, the greater the character distance, the greater the difference between the input text and each text to be processed.

[0216] In this way, by comparing the magnitudes of the edit distances calculated by the above various methods, the minimum edit distance between the input text and each text to be processed can be calculated, and furthermore, the optimal editing method for converting the input text to the text to be processed can be determined.

[0217] Step 1019: Calculate the character distance between the input text and each text to be processed according to the number of identical phrases existing between the input text and each text to be processed.

[0218] Optionally, the character distance is used to characterize the difference between the input text and each text to be processed. Generally, the greater the character distance, the greater the difference between the input text and each text to be processed.

[0219] Optionally, the character distance may be calculated according to the following formula (3).

[0220]

[0221] Among them, d g is the character distance, S g (A, B) is the distance value between the input text and each text to be processed, A is the input text, B is the text to be processed, |A∩B| is the number of identical phrases between the input text and a text to be processed, |A| is the number of fragments in the input text, and |B| is the number of fragments of a text to be processed.

[0222] Step 1020: Calculate the difference value according to the character distance and the minimum edit distance.

[0223] Optionally, a weighted sum is performed on the character distance and the minimum edit distance, and the obtained sum is used as the difference value.

[0224] Specifically, the difference value can be calculated according to the following formula (4).

[0225] d=w1d s +w2d g (4)

[0226] Among them, d is the difference value, w1 is the first weight, w2 is the second weight, d s is the character distance, d g is the minimum edit distance.

[0227] Optionally, the first weight and the second weight can be values ​​set in advance by relevant technicians according to actual needs. Generally, the first weight can be set to 0.2 and the second weight can be set to 0.8. Of course, the first weight and the second weight can also be set to any other possible values. This embodiment of the application is not limited to this.

[0228] In this way, the accuracy of the calculated difference value can be improved, and the accuracy of determining the similarity or approximation between the input text and each to-be-processed text can also be improved. Then, the standard text is output according to the calculated difference value, so that the difference between the output standard text and the input text can be reduced, thereby achieving the effect of accuracy in file processing.

[0229] In order to avoid duplication among the acquired texts to be processed, a deduplication operation may be performed on the acquired texts to be processed.

[0230] Specifically, the deduplication operation may be performed in the following manner. In a possible implementation manner, each to-be-processed text is determined according to each first text, each second text, and each third text, including:

[0231] A to-be-processed text library is generated according to the first text, the second text and the third text.

[0232] Optionally, the to-be-processed text library may include all first texts, all second texts, and all third texts.

[0233] Determine whether there is duplicate text in the text library to be processed.

[0234] Optionally, the to-be-processed text library may be traversed, and any text in the to-be-processed text library may be compared with other texts in the to-be-processed text library to determine whether there is a duplicate text.

[0235] If so, the other texts in the to-be-processed text library except the repeated text are used as the to-be-processed texts.

[0236] Specifically, if it is determined that there are duplicate texts in the to-be-processed text library, that is, if it is determined that any two texts in the to-be-processed text library are identical, any one of the two identical texts is deleted, and only one of the two identical texts is retained.

[0237] In this way, the deduplication operation can be completed, and the situation where the obtained texts to be processed are repeated can be avoided.

[0238] If it does not exist, all the texts in the to-be-processed text library are used as the to-be-processed texts.

[0239] In this way, it is shown that each first text, each second text and each third text are different, and each first text, each second text and each third text can be used as the text to be processed. In this way, the flexibility of determining the text to be processed can be improved, and the situation where the obtained texts to be processed are repeated can be avoided, thereby reducing the pressure of determining the difference value and outputting the standard text in the subsequent process.

[0240] In a possible implementation, outputting a standard text matching the input text according to each difference value includes:

[0241] The text to be processed corresponding to the smallest difference value among the difference values ​​is used as the standard text.

[0242] Output the standard text.

[0243] In a possible implementation, before obtaining at least one word segmentation result of the input text, the method further includes:

[0244] Perform preprocessing operations on the input text.

[0245] Optionally, the preprocessing operation includes at least one of the following: converting all English characters in the input text into lowercase characters, converting traditional Chinese characters in the input text into simplified Chinese characters, deleting special symbols in the input text, and correcting typos in the input text.

[0246] In this way, the input text can be segmented more accurately to obtain the segmentation results, and the accuracy of determining the above-mentioned difference value can be improved, thereby reducing the difference between the output standard text and the input text, thereby achieving the effect of accurate file processing.

[0247] The following describes the apparatus, device, and computer-readable storage medium used to execute the text processing method provided in the present application. The specific implementation process and technical effects are described above and will not be repeated below.

[0248] Figure 8 is a structural diagram of a text processing device provided in an embodiment of the present application, see Fig. 9 , the device comprises:

[0249] The first acquisition module 201 is used to acquire at least one word segmentation result of the input text.

[0250] Optionally, the input text may be a text including address information, a text including case information, or a text including any possible information, which is not limited in the embodiment of the present application.

[0251] Optionally, each word segmentation result includes at least one type of phrase in the input text.

[0252] Optionally, each type may include an address, a person's name, an administrative division name, a disease name, a drug name, etc.

[0253] The second acquisition module 202 is used to acquire at least one to-be-processed text in the standard text database according to each word segmentation result.

[0254] Optionally, the standard text database is used to store multiple standardized texts, and each standardized text is used to identify address information.

[0255] The determination module 203 is used to respectively determine the difference values ​​between the input text and each to-be-processed text.

[0256] Optionally, the difference value may be used to characterize the similarity or approximation between the input text and the text to be processed.

[0257] The output module 204 is used to output the standard text matching the input text according to the difference values.

[0258] Optionally, the standard text matching the input text may be a text to be processed that has the smallest difference value with the input text among the texts to be processed. And since each text to be processed is obtained by segmenting each standard text stored in the standard text database into a plurality of standard phrase combinations, each text to be processed is also a standardized text.

[0259] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0260] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0261] Fig. 9 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig. 9 The computer device includes: a memory 301 and a processor 302. The memory 301 stores a computer program that can be run on the processor 302. When the processor 302 executes the computer program, the steps in any of the above method embodiments are implemented.

[0262] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0263] Optionally, the present application also provides a program product, such as a computer-readable storage medium, including a program, which is used to execute any of the above-mentioned text processing method embodiments when executed by a processor.

[0264] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0265] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0266] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0267] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, referred to as: ROM), random access memory (English: Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0268] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0269] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A text processing method, It is characterized in that include: Obtaining at least one word segmentation result of the input text, each of the word segmentation results including at least one type of phrase in the input text; Acquire at least one text to be processed in a standard text database according to each of the word segmentation results, wherein the standard text database is used to store a plurality of standardized texts, each of which is used to identify address information; respectively determining the difference values ​​between the input text and each of the to-be-processed texts; Outputting a standard text matching the input text according to each of the difference values; The step of acquiring at least one text to be processed in a standard text database according to each of the word segmentation results comprises: According to each of the word segmentation results and a preset order, sequentially acquiring at least one first text matching each of the word segmentation results from the standard text database; Obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order; Performing a completion process on each of the word segmentation results to obtain a completed word segmentation result, and acquiring at least one third text matching the updated word segmentation result from the standard text database in sequence according to the completed word segmentation result and the preset order; The to-be-processed texts are determined according to the first texts, the second texts and the third texts.

2. The text processing method according to claim 1, It is characterized in that The step of acquiring at least one text to be processed in a standard text database according to each of the word segmentation results comprises: Traversing the standard text database according to each of the word segmentation results, obtaining at least one standard phrase matching each of the word segmentation results, wherein the standard phrase is a phrase obtained by segmenting each of the standardized texts; The at least one text to be processed is generated according to each of the standard phrases.

3. The text processing method according to claim 1, It is characterized in that The step of obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order includes: Obtaining a changed address corresponding to the address indicated by each segmentation result from a preset address change database, and updating the segmentation result based on the changed address to obtain an updated segmentation result; According to the updated word segmentation result and the preset order, at least one second text matching the updated word segmentation result is acquired from the standard text database in sequence.

4. The text processing method according to claim 1, It is characterized in that The determining each of the to-be-processed texts according to each of the first texts, each of the second texts, and each of the third texts comprises: Generate a text library to be processed according to the first text, the second text and the third text; Determine whether there is duplicate text in the text library to be processed; If so, other texts in the to-be-processed text library except the repeated texts are used as the to-be-processed texts; If it does not exist, all the texts in the to-be-processed text library are used as the to-be-processed texts.

5. The text processing method according to claim 1, It is characterized in that The step of respectively determining the difference value between the input text and each of the to-be-processed texts comprises: Determining the minimum number of single-character edits required to convert the input text into the text to be processed; Determining the number of identical phrases between the input text and each of the to-be-processed texts; The difference value is calculated based on the minimum number of single-character edits and the number of identical phrases.

6. The text processing method according to claim 5, It is characterized in that The calculating the difference value according to the minimum number of single-character edits and the number of identical phrases includes: Calculating the minimum edit distance between the input text and each of the to-be-processed texts according to the minimum number of single-character edits required to convert the input text into the to-be-processed texts; Calculating the character distance between the input text and each of the to-be-processed texts according to the number of identical phrases existing between the input text and each of the to-be-processed texts; The difference value is calculated according to the character distance and the minimum edit distance.

7. A text processing device, It is characterized in that The device comprises: A first acquisition module, used to acquire at least one word segmentation result of an input text; A second acquisition module, used for acquiring at least one text to be processed in a standard text database according to each of the word segmentation results; A determination module, used for respectively determining the difference value between the input text and each of the to-be-processed texts; An output module, used for outputting a standard text matching the input text according to each of the difference values; The second acquisition module is specifically used for: According to each of the word segmentation results and a preset order, sequentially acquiring at least one first text matching each of the word segmentation results from the standard text database; Obtaining an updated word segmentation result based on the changed address corresponding to the address indicated by each word segmentation result, and acquiring at least one second text matching the updated word segmentation result from the standard text database according to the updated word segmentation result and the preset order; Performing a completion process on each of the word segmentation results to obtain a completed word segmentation result, and acquiring at least one third text matching the updated word segmentation result from the standard text database in sequence according to the completed word segmentation result and the preset order; The to-be-processed texts are determined according to the first texts, the second texts and the third texts.

8. A computer device, It is characterized in that include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.