Address text processing method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses an address text processing method and device, equipment, a medium and a product, and the method comprises the steps: determining a first address element recognition result of a to-be-processed address text based on a preset address element dictionary, determining a second address element recognition result of the to-be-processed address text based on a target address element recognition model obtained by training based on a Bert language model, and determining a target address element combination in the first address element recognition result and the second address element recognition result based on path planning; and obtaining a target address text corresponding to the to-be-processed address text. According to the technical scheme, the problem that an existing address information processing result is not accurate is solved, the comprehensiveness of address matching can be guaranteed through a dictionary by combining a dictionary matching technology and a Bert language model, meanwhile, the understanding of context semantic association of the to-be-processed address data can be enhanced through the Bert model, and the processing efficiency of the to-be-processed address data is improved. And the accuracy of address standardization is ensured.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of computer vision, and in particular to an address text processing method, device, equipment, medium and product. Background Art

[0002] Currently, address information is processed through dictionaries to convert non-standardized address information into standardized address information, or processed through neural network models such as GPT (generative pre-trained transformer), Transformer or RNN (recurrent neural network). However, these methods often fail to achieve good standardization processing effects and cannot effectively correct the address information. Summary of the invention

[0003] The embodiments of the present invention provide an address text processing method, apparatus, device, medium and product. By combining dictionary matching technology and the Bert language model, the comprehensiveness of address matching can be ensured through the dictionary, and the understanding of the contextual semantic association of the address data to be processed can be enhanced through the Bert model to ensure the accuracy of address standardization.

[0004] In a first aspect, an embodiment of the present invention provides an address text processing method, the method comprising:

[0005] Get the address text to be processed;

[0006] Determine a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on a Bert language model;

[0007] Perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result;

[0008] The order of each address element in the target address element combination is adjusted based on the address hierarchical relationship rule to obtain the target address text corresponding to the address text to be processed.

[0009] In a second aspect, an embodiment of the present invention provides an address text processing device, the device comprising:

[0010] A module for obtaining text to be processed, used to obtain address text to be processed;

[0011] A text recognition module, which determines a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determines a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on a Bert language model;

[0012] a target address element determination module, configured to perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine a target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result;

[0013] The target address text determination module is used to adjust the order between each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0015] one or more processors;

[0016] A memory for storing one or more programs;

[0017] When the one or more programs are executed by one or more processors, the one or more processors implement the address text processing method provided by any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an address text processing method as provided in any embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the address text processing method provided by any embodiment of the present invention.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] In the embodiment of the present invention, the address text to be processed is obtained;

[0022] Determine the first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine the second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on the Bert language model; perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; adjust the order between each address element in the target address element combination based on the address hierarchical relationship rule, and obtain the target address text corresponding to the address text to be processed. The technical solution of the embodiment of the present invention solves the problem of inaccurate address information processing results at present, and can combine the dictionary matching technology and the Bert language model, while ensuring the comprehensiveness of address matching through the dictionary, and can also enhance the understanding of the contextual semantic association of the address data to be processed through the Bert model to ensure the accuracy of address standardization. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of an address text processing method provided by an embodiment of the present invention;

[0024] Figure 2 is a flow chart of an address text processing method provided by an embodiment of the present invention;

[0025] Figure 3 is a flow chart of an address text processing method provided by an embodiment of the present invention;

[0026] Figure 4 is a flow chart of an address text processing method provided by an embodiment of the present invention;

[0027] Figure 5 is a structural schematic diagram of an address text processing device provided by an embodiment of the present invention;

[0028] Figure 6 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0030] Figure 1This is a flowchart of an address text processing method provided by an embodiment of the present invention. This embodiment is applicable to the scene of address information processing. The method can be executed by an address text processing device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0031] like Figure 1 As shown, the address text processing method of this embodiment includes the following steps:

[0032] S110, obtaining the address text to be processed.

[0033] The address text to be processed may be the address text to be processed entered by the user in the platform or the browser. Specifically, it may be a non-standardized address text, such as a common name or an address lacking a standard format, or an address text having spelling errors or missing characters.

[0034] S120, determining a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determining a second address element recognition result of the address text to be processed based on a target address element recognition model.

[0035] The preset address element dictionary may include a Chinese character dictionary dataset and a pinyin dictionary dataset, which is a data set composed of address elements. Address elements are basic components used to accurately describe geographic location information, such as components corresponding to geographic levels, such as components corresponding to provincial level, city level, district and county level, and street level in standardized address text.

[0036] After the address text to be processed is segmented, it is matched and identified with the address elements in the preset address element dictionary to obtain the address elements that match each word in the address text to be processed, and the first address element recognition result is obtained. This embodiment can quickly match and identify various place names and addresses by establishing a comprehensive Chinese character dictionary data set and a pinyin dictionary data set, especially for address text to be processed with inconsistent spelling, it has a good error correction effect, and can simultaneously input the address text to be processed into the target address element recognition model, and perform corresponding standardized address text prediction on the address text to be processed through the target address element recognition model to obtain the second address element recognition result of the address text to be processed.

[0037] Specifically, the target address element recognition model can automatically correct errors and complete the address text to be processed based on the address text features learned in the training phase, including automatically identifying and correcting typos and spelling errors in the address, and completing missing address elements to obtain a second address element recognition result. The target address element recognition model can be trained based on the Bert language model. The Bert language model is an advanced natural language processing model. In this embodiment, the target address element recognition model is obtained by training the Bert language model to improve the context parsing ability of the target address element recognition model, thereby understanding and processing the complex place name address structure in the address text to be processed, and standardizing the address text to be processed.

[0038] S130. Perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine a target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result.

[0039] In this embodiment, dynamic path planning is performed on the address elements, for example, corresponding path planning is performed on the address elements in the first address element recognition result and the second address element recognition result through the Viterbi algorithm, that is, the address elements in the standardized address text that are most likely to correspond to the address text to be processed, as well as the arrangement order of these address elements, are determined, thereby obtaining the target address element combination.

[0040] S140: Adjust the order of each address element in the target address element combination based on the address hierarchy relationship rule to obtain a target address text corresponding to the address text to be processed.

[0041] It is understandable that the target address element combination may have address arrangement errors. Therefore, in order to ensure the accuracy of the final target address text in address logic, the order between each address element in the target address element combination is adjusted according to the address hierarchy relationship rules to obtain the target address text corresponding to the address text to be processed and in compliance with the address hierarchy relationship rules.

[0042] Of course, in some address text application situations, the order between each address element in the target address element combination may also be adjusted according to a preset address arrangement rule to obtain a target address text corresponding to the address text to be processed.

[0043] The technical solution of this embodiment is to obtain the address text to be processed; determine the first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine the second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on the Bert language model; perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; adjust the order between each address element in the target address element combination based on the address hierarchical relationship rule, and obtain the target address text corresponding to the address text to be processed. The technical solution of the embodiment of the present invention solves the problem of inaccurate address information processing results at present, and can combine the dictionary matching technology and the Bert language model, while ensuring the comprehensiveness of address matching through the dictionary, and can also enhance the understanding of the contextual semantic association of the address data to be processed through the Bert model to ensure the accuracy of address standardization.

[0044] Figure 2 This is a flowchart of an address text processing method provided in an embodiment of the present invention. This embodiment and the address text processing method in the above embodiment belong to the same inventive concept, and further describes the process of determining the recognition result of the first address element. The method can be executed by an address text processing device, which can be implemented by software and / or hardware and integrated in a computer device with application development function.

[0045] like Figure 2 As shown, the address text processing method of this embodiment includes the following steps:

[0046] S210: Obtain the address text to be processed.

[0047] S220, performing dictionary matching on the Chinese character dictionary data set based on the preset address element dictionary and the address text to be processed, and obtaining a Chinese character dictionary matching result of the address text to be processed.

[0048] The process of establishing a Chinese character dictionary data set may be to obtain standard place name information, and further obtain the common names and historical names corresponding to the standard place name information, and form a Chinese character dictionary data set based on the collected information. The Chinese character dictionary data set includes common names and historical names to support a wider range of address recognition. Optionally, the Chinese character dictionary data set also includes POI (Point of Interest) dictionary data. The POI dictionary can contain public point information, such as shopping malls, schools, etc., which are usually frequently appearing references in colloquial addresses. By using public point information as one of the bases for matching, the comprehensiveness of matching is further improved. It is thus applied to the processing of big data address texts, and users may be more inclined to use well-known POIs around them as references when providing address information. The use of such a dictionary can better meet user habits and improve the user-friendliness of the analysis.

[0049] Based on the Chinese character dictionary data set of the preset address element dictionary, the dictionary matching is performed with the address text to be processed. For example, the Chinese character dictionary matching result of the address text to be processed is obtained by applying fuzzy matching and other methods based on the words obtained by word segmentation in the address text to be processed. Fuzzy matching can support the recognition of synonyms and homophones, and improve the accuracy and comprehensiveness of recognition.

[0050] This embodiment can quickly match and identify various place names and addresses by establishing a comprehensive Chinese and Pinyin dictionary, especially when faced with inconsistent spelling or slight errors. The dual dictionary structure of Pinyin and Chinese characters enables address resolution to overcome the barriers of language input. For example, when a user uses the Pinyin input method to input an address, the system can still accurately match it, improving the flexibility of matching.

[0051] S230, performing pinyin conversion on the address text to be processed, performing dictionary matching on the pinyin dictionary data set based on the preset address element dictionary and the address text to be processed after pinyin conversion, and obtaining a dictionary matching result in pinyin form.

[0052] The process of establishing the pinyin dictionary data set of the preset address element dictionary can be to convert all address elements of the Chinese character dictionary data set, that is, all Chinese place names and variants, into non-tone pinyin forms in order to process pinyin input and support pinyin correction mechanism.

[0053] S240, converting the dictionary matching result in pinyin form into Chinese character form to obtain a pinyin dictionary matching result.

[0054] The dictionary matching results in pinyin form are converted back to Chinese characters through the mapping from pinyin to Chinese, so as to obtain the pinyin dictionary matching results in Chinese character form.

[0055] In an optional implementation, the dictionary matching results in Pinyin form are converted into Chinese character form to obtain Pinyin dictionary matching results. Each address element in the Pinyin dictionary matching results can be matched with the address element in the Chinese character dictionary data set, and the address elements matched by the Pinyin dictionary matching results in the Chinese character dictionary data set are used as the Pinyin dictionary matching results.

[0056] It can be understood that all the contents in the Pinyin dictionary data set are converted from the Chinese character dictionary data set. Therefore, by matching each address element in the Pinyin dictionary matching result with the address element in the Chinese character dictionary data set word by word, the corresponding Chinese character address element can be obtained, thereby obtaining the Pinyin dictionary matching result composed of the matched Chinese character address elements.

[0057] S250: Determine a second address element recognition result of the address text to be processed based on a target address element recognition model.

[0058] S260: Perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine a target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result.

[0059] Specifically, in path planning, the pinyin dictionary matching results and the Chinese character dictionary matching results in the first address element recognition results are taken as separate path planning sets, together with the address elements in the second address element recognition results, to obtain three path planning sets, and path planning is performed on each address element in the three path planning sets to determine the target address element combination.

[0060] S270: Adjust the order of each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed.

[0061] The technical solution of this embodiment is to obtain the address text to be processed; perform dictionary matching on the address text to be processed based on a Chinese character dictionary data set of a preset address element dictionary to obtain a Chinese character dictionary matching result of the address text to be processed, and perform pinyin conversion on the address text to be processed, perform dictionary matching on the pinyin dictionary data set of a preset address element dictionary and the address text to be processed after pinyin conversion to obtain a dictionary matching result in pinyin form; convert the dictionary matching result in pinyin form into Chinese character form to obtain a pinyin dictionary matching result; determine a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on the Bert language model; perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; adjust the order between each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed. The technical solution of the embodiment of the present invention solves the problem of inaccurate address information processing results. It can ensure the comprehensiveness of address matching by combining dictionary matching technology and Bert language model, and match the text to be processed through dictionaries in the form of Chinese characters and pinyin. It can handle a wider range of scenarios and input variants, and improve the robustness and accuracy of address text processing.

[0062] Figure 3 This is a flowchart of an address text processing method provided by an embodiment of the present invention. This embodiment and the address text processing method in the above embodiment belong to the same inventive concept, and further describes the process of path planning to determine the combination of target address elements. The method can be executed by an address text processing device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0063] like Figure 3 As shown, the address text processing method of this embodiment includes the following steps:

[0064] S310: Obtain the address text to be processed.

[0065] S320: Determine a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine a second address element recognition result of the address text to be processed based on a target address element recognition model.

[0066] S330. Determine a time step value in path planning according to the number of address elements in the first address element recognition result and / or the second address element recognition result.

[0067] The path planning of the address elements in the recognition results is performed by the Viterbi algorithm. Specifically, a hidden Markov model is established to determine the hidden state and observation sequence. The standard address element categories are set as hidden states, such as province, city, district, street, house number, etc. as different hidden states. The address elements in the first address element recognition result and the second address element recognition result are used as observation sequences.

[0068] The time step value in path planning is determined according to the number of address elements in the first address element recognition result and / or the second address element recognition result, i.e., the total number of time steps. The time step value corresponds to the number of address elements in the standardized address text of the address text to be processed, i.e., the target address text.

[0069] S340, determining the initial state probability of the address elements in the first address element recognition result and the second address element recognition result in the path planning, and the initial state probability is used to determine the target initial address element of the first time step of the path planning.

[0070] For example, the initial state probabilities of the address elements in the first address element recognition result and the second address element recognition result in the path planning are set according to prior knowledge.

[0071] In an optional embodiment, the initial state probability of the address element in the second address element recognition result is higher than the initial state probability of the address element in the first address element recognition result, thereby increasing the probability that the model recognition result, i.e., the second address element recognition result, appears in the final target address element combination. Because the recognition result of the Bert model has the coherence of contextual semantics, it is more accurate than the dictionary matching result.

[0072] S350. At each time step, determine the previous address element corresponding to the maximum path probability to each address element, and obtain the target address element corresponding to each time step.

[0073] Among them, the target address element determined in the last time step is the target final address element.

[0074] At each time step, the previous address element corresponding to the maximum path probability to reach each address element is determined by recursive calculation, and the target address element corresponding to each time step is obtained. That is, for each time point of path planning, the address element is regarded as the state of the Viterbi algorithm, and for each state, the maximum path probability of reaching the state through all possible previous states is calculated.

[0075] After processing all inputs, the state with the highest probability at the last time point is selected as the end state, that is, the target address element determined in the last time step is the target final address element.

[0076] S360, starting from the target final address element, tracing back to the target initial address element, recording each target address element between the target initial address element and the target final address element, and obtaining a target address element combination.

[0077] Starting from the final state, trace back to the initial state, and record each intermediate state between the final state and the initial state, and record the state sequence that forms the optimal path. The state sequence of the optimal path is the state sequence that most likely represents the actual address, that is, the target address element combination.

[0078] S370: Adjust the order of each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed.

[0079] The technical solution of this embodiment is to obtain the address text to be processed; determine the first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine the second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training the Bert language model; determine the time step value in the path planning according to the number of address elements in the first address element recognition result and / or the second address element recognition result; determine the initial state probability of the address elements in the first address element recognition result and the second address element recognition result in the path planning, and the initial state probability is used to determine the target initial address element of the first time step of the path planning; at each time step, determine the previous address element corresponding to the maximum path probability of reaching each address element, and obtain the target address element corresponding to each time step, wherein the target address element determined in the last time step is the target final address element; starting from the target final address element, reversely trace back to the target initial address element, record each target address element between the target initial address element and the target final address element, and obtain the target address element combination. Adjust the order between each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed. The technical solution of the embodiment of the present invention solves the problem of inaccurate address information processing results. It can optimize the selection of the most likely correct address by matching the recognition results with the Bert language model through a dictionary, and integrating these matching results through a path planning algorithm, which not only improves the parsing accuracy, but also reduces errors and ambiguities.

[0080] Figure 4This is a flowchart of an address text processing method provided by an embodiment of the present invention. This embodiment and the address text processing method in the above embodiment belong to the same inventive concept, and further describes the training process of the target address element recognition model. The method can be executed by an address text processing device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0081] like Figure 4 As shown, the training process of the target address element recognition model in the address text processing method of this embodiment includes the following steps:

[0082] S410: Acquire sample address texts in the pre-training phase and sample address texts in the fine-tuning phase.

[0083] In this embodiment, the training process of the target address element recognition model includes two stages, namely, a pre-training stage and a fine-tuning stage. Diverse text data is collected as pre-trained sample address texts, for example, a large number of user-annotated addresses are collected from map software, and addresses obtained from geographic information databases, etc.

[0084] In addition to formal address texts, the sample address texts in the pre-training stage can also include colloquial address expressions with variants to enhance the model's adaptability to different forms of expression.

[0085] The sample address text in the fine-tuning stage can be a non-standardized address text. For example, fine-tuning data specifically used for address text standardization is collected as the sample address text in the fine-tuning stage. For example, text containing address errors or non-standard expressions and the corresponding standard address text are collected to form a sample data pair.

[0086] S420. Pre-train the Bert language model using the sample address texts in the pre-training phase to obtain a pre-trained address element recognition model.

[0087] Perform named entity annotation on the sample address text in the pre-training phase. The annotated entity type can be determined according to the address level to which the address element belongs, such as country, province, city, district, street, community, neighborhood, building name, and landmark name.

[0088] By setting model learning tasks, such as masked language model tasks and next sentence prediction (NSP) tasks, the Bert language model is pre-trained based on the sample address text in the pre-training phase to obtain a pre-trained address element recognition model.

[0089] S430. Fine-tune the pre-trained address element recognition model using the sample address text in the fine-tuning stage to obtain a target address element recognition model.

[0090] According to the goal of address text standardization, fine-tuning tasks are designed, such as correction of named entity annotations, adjustment of address element order, and normalization of non-standard expressions. Through the sample address text in the fine-tuning stage, the pre-trained address element recognition model is fine-tuned based on the preset fine-tuning tasks to obtain the target address element recognition model.

[0091] The technical solution of this embodiment is to obtain sample address texts in the pre-training stage and sample address texts in the fine-tuning stage; pre-train the Bert language model through the sample address texts in the pre-training stage to obtain a pre-trained address element recognition model; fine-tune the pre-trained address element recognition model through the sample address texts in the fine-tuning stage to obtain a target address element recognition model; wherein the sample address texts in the fine-tuning stage are non-standardized address texts. The technical solution of the embodiment of the present invention solves the problem of inaccurate address information processing results at present, and can enhance the understanding of the contextual semantic association of the address data to be processed through the Bert model with good context parsing ability, thereby ensuring the accuracy of address standardization.

[0092] Figure 5 The present invention provides a schematic diagram of the structure of an address text processing device, which is applicable to the scene of address information processing. The address text processing device can be implemented by software and / or hardware, and integrated into a computer terminal device with application development function.

[0093] like Figure 5 As shown, the address text processing device includes: a to-be-processed text acquisition module 510, a text recognition module 520, a target address element determination module 530 and a target address text determination module 540.

[0094] Among them, the module for obtaining the text to be processed 510 is used to obtain the address text to be processed; the text recognition module 520 determines the first address element recognition result of the address text to be processed based on a preset address element dictionary, and determines the second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained based on the training of the Bert language model; the target address element determination module 530 is used to perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; the target address text determination module 540 is used to adjust the order between each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed.

[0095] The technical solution of this embodiment is to obtain the address text to be processed; determine the first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine the second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on the Bert language model; perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; adjust the order between each address element in the target address element combination based on the address hierarchical relationship rule, and obtain the target address text corresponding to the address text to be processed. The technical solution of the embodiment of the present invention solves the problem of inaccurate address information processing results at present, and can combine the dictionary matching technology and the Bert language model, while ensuring the comprehensiveness of address matching through the dictionary, and can also enhance the understanding of the contextual semantic association of the address data to be processed through the Bert model to ensure the accuracy of address standardization.

[0096] In an optional implementation, the text recognition module 520 is specifically used for:

[0097] Dictionary matching is performed between a Chinese character dictionary data set based on a preset address element dictionary and the address text to be processed to obtain a Chinese character dictionary matching result of the address text to be processed; and, the address text to be processed is converted to pinyin, and a pinyin dictionary data set based on a preset address element dictionary is matched with the address text to be processed after pinyin conversion to obtain a dictionary matching result in pinyin form; and the dictionary matching result in pinyin form is converted into Chinese character form to obtain a pinyin dictionary matching result.

[0098] In an optional implementation, the text recognition module 520 is further configured to:

[0099] Each address element in the pinyin dictionary matching result is matched with the address element in the Chinese dictionary data set, and the address element matched by the pinyin dictionary matching result in the Chinese dictionary data set is used as the pinyin dictionary matching result.

[0100] In an optional implementation manner, the target address element determination module 530 is specifically configured to:

[0101] Determine the time step value in the path planning according to the number of address elements in the first address element recognition result and / or the second address element recognition result; determine the initial state probability of the address elements in the first address element recognition result and the second address element recognition result in the path planning, and the initial state probability is used to determine the target initial address element of the first time step of the path planning; at each time step, determine the previous address element corresponding to the maximum path probability to reach each address element, and obtain the target address element corresponding to each time step, wherein the target address element determined in the last time step is the target final address element; starting from the target final address element, trace back to the target initial address element, record each target address element between the target initial address element and the target final address element, and obtain the target address element combination.

[0102] In an optional implementation, the initial state probability of the address element in the second address element recognition result is higher than the initial state probability of the address element in the first address element recognition result.

[0103] In an optional embodiment, the device further comprises:

[0104] The model training module is used to obtain sample address texts in the pre-training stage and sample address texts in the fine-tuning stage; pre-train the Bert language model through the sample address texts in the pre-training stage to obtain a pre-trained address element recognition model; fine-tune the pre-trained address element recognition model through the sample address texts in the fine-tuning stage to obtain a target address element recognition model; wherein the sample address texts in the fine-tuning stage are non-standardized address texts.

[0105] The address text processing device provided in the embodiment of the present invention can execute the address text processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0106] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller and server, a mobile phone and other terminal devices.

[0107] like Figure 6As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0108] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0109] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0110] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0111] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0112] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RFID systems, tape drives, and data backup storage systems.

[0113] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the address text processing method provided in the embodiment of the present invention, which includes:

[0114] Get the address text to be processed;

[0115] Determine a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the address element recognition model is obtained by training based on a Bert language model;

[0116] Perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result;

[0117] The order of each address element in the target address element combination is adjusted based on the address hierarchical relationship rule to obtain the target address text corresponding to the address text to be processed.

[0118] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the address text processing method provided by any embodiment of the present invention is implemented. The method includes:

[0119] Get the address text to be processed;

[0120] Determine a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the address element recognition model is obtained by training based on a Bert language model;

[0121] Perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine the target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result;

[0122] The order of each address element in the target address element combination is adjusted based on the address hierarchical relationship rule to obtain the target address text corresponding to the address text to be processed.

[0123] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0124] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0125] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0126] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, Python, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the address text processing method provided in any embodiment of the present application.

[0128] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, Python, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0129] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0130] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for processing address text, characterized in that: include: Get the address text to be processed; Determine a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determine a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on a Bert language model; Performing corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determining a target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; The order of each address element in the target address element combination is adjusted based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed.

2. The method according to claim 1, characterized in that The determining the first address element recognition result of the address text to be processed based on a preset address element dictionary includes: Performing dictionary matching on the Chinese character dictionary data set based on the preset address element dictionary and the address text to be processed to obtain a Chinese character dictionary matching result of the address text to be processed, and, Performing pinyin conversion on the address text to be processed, performing dictionary matching on the pinyin dictionary data set of the preset address element dictionary and the address text to be processed after pinyin conversion, and obtaining a dictionary matching result in pinyin form; The dictionary matching result in pinyin form is converted into Chinese character form to obtain a pinyin dictionary matching result.

3. The method according to claim 2, characterized in that The step of converting the dictionary matching result in pinyin form into Chinese character form to obtain the pinyin dictionary matching result includes: Each address element in the dictionary matching result in pinyin form is matched with the address element in the Chinese character dictionary data set, and the address element matched by the dictionary matching result in pinyin form in the Chinese character dictionary data set is used as the pinyin dictionary matching result.

4. The method according to claim 1, characterized in that: The performing corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determining a target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result, comprises: Determine a time step value in path planning according to the number of address elements in the first address element recognition result and / or the second address element recognition result; Determine the initial state probability of the address elements in the first address element recognition result and the second address element recognition result in the path planning, wherein the initial state probability is used to determine the target initial address element of the first time step of the path planning; At each time step, determine the previous address element corresponding to the maximum path probability to each of the address elements, and obtain the target address element corresponding to each time step, wherein the target address element determined in the last time step is the target final address element; Starting from the target final address element, tracing back to the target initial address element, recording each target address element between the target initial address element and the target final address element, and obtaining a target address element combination.

5. The method according to claim 4, characterized in that The initial state probability of the address element in the second address element recognition result is higher than the initial state probability of the address element in the first address element recognition result.

6. The method according to any one of claims 1 to 5, characterized in that: The training process of the target address element recognition model includes: Obtain sample address texts in the pre-training phase and sample address texts in the fine-tuning phase; Pre-training the Bert language model using the sample address texts in the pre-training phase to obtain a pre-trained address element recognition model; Fine-tune the pre-trained address element recognition model using the sample address text in the fine-tuning stage to obtain a target address element recognition model; The sample address text in the fine-tuning stage is a non-standardized address text.

7. An address text processing device, characterized in that: include: A module for obtaining text to be processed, used to obtain address text to be processed; A text recognition module, which determines a first address element recognition result of the address text to be processed based on a preset address element dictionary, and determines a second address element recognition result of the address text to be processed based on a target address element recognition model, wherein the target address element recognition model is obtained by training based on a Bert language model; a target address element determination module, configured to perform corresponding path planning according to the address elements in the first address element recognition result and the second address element recognition result, and determine a target address element combination in the first address element recognition result and the second address element recognition result based on the path planning result; The target address text determination module is used to adjust the order between each address element in the target address element combination based on the address hierarchy relationship rule to obtain the target address text corresponding to the address text to be processed.

8. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the address text processing method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the address text processing method as described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the address text processing method as described in any one of claims 1 to 7.

Images