Address similarity calculation method, device, equipment and storage medium

CN118296405BActive Publication Date: 2024-11-05SHENZHEN AEROSPACE SMART CITY SYST TECH RES INST CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410719381.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-05
Publication Date
2024-11-05
Estimated Expiration
2044-06-05

AI Technical Summary

Technical Problem

但是现有技术均存在以下缺点:准确率不高,只能通过单一字段进行简单相似度的计算,没有考虑到多个字段的关联关系;长文本支持效果不好,难以从长文本中准确提取地址的关键信息

Benefits of technology

[0026] Embodiments of the present disclosure provide an address similarity calculation method, device and storage medium to solve the problem that multiple fields have valid addresses or multiple fields need to be integrated to confirm valid addresses. The problem of low matching success rate; and the problem of low accuracy of effective address extraction from long text, the method of the present invention utilizes The BERT model extracts corpus features from the input multi-field address data, and converts the corpus features into text feature vectors of structured standard addresses; using the structured standard addresses as samples, a dynamically updated standard address is established library; according to the importance of different fields in the address and the characteristics of the BERT model training process Feature representation, set corresponding weights for each field, so as to comprehensively consider the influence of each part in the similarity calculation stage; use the cosine similarity method to calculate the similarity of the Chinese text and its corresponding pinyin based on the text feature vector, for Measuring the similarity between addresses in two dimensions; determining multi-character based on the similarity between Chinese text and its corresponding pinyin comprehensive similarity of text segments, and determine whether two multi-field texts are similar based on a preset threshold; dynamically updating the standard address library based on the similarity results between addresses, its beneficial effect not only significantly improves the performance of multi-field text success rate in address matching tasks, and effectively solves the problem of low accuracy in extracting effective addresses from long texts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118296405B_ABST
    Figure CN118296405B_ABST
Patent Text Reader

Abstract

The invention discloses an address similarity calculation method, device and storage medium. The method comprises: using a BERT model to extract corpus features from multi-field address data and converting them into text feature vectors of structured standard addresses; setting corresponding weights for each field according to the importance of different fields in the address and the feature performance in BERT model training, so as to comprehensively consider the influence of each part in the similarity calculation stage; using a cosine similarity method to calculate the similarity of Chinese text and its corresponding pinyin according to the text feature vector, and measuring the similarity between addresses in two dimensions; based on the similarity of Chinese text and its corresponding pinyin, determining the comprehensive similarity of multi-field texts, and setting a threshold to judge whether two multi-field texts are similar; the method of the invention not only improves the success rate when facing multi-field address matching tasks, but also effectively solves the problem of low accuracy in extracting effective addresses in long texts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of data matching, and in particular relates to an address similarity calculation method, device, equipment and storage medium. Background Art

[0002] Address is the most commonly used basic information in logistics and letter business. At present, letter addresses are generally filled in manually, and there are problems such as inconsistent description format, non-standard address names and typos. Especially when there are duplicate letters, the same letter will be processed repeatedly because the letters are not compared in time, which seriously affects the efficiency of letter processing. Some existing technologies will compare address information. The commonly used address matching algorithm is to use the Chinese word segmentation algorithm, set dictionary rules to extract various place names, and then combine the word segmentation features to calculate the similarity between addresses. Specifically, the specific implementation of address matching in the existing technical solution is as follows: first, the text description address entered by the user is segmented; secondly, for each segmentation, the segmentation is matched with the pre-stored segmentation in the segmentation library to obtain at least one pre-stored segmentation matching the segmentation; thirdly, according to the word order of each segmentation in the address to be matched, the pre-stored segmentations corresponding to each segmentation are combined to obtain multiple combinations; various combination results are displayed to the user for the user to choose, and address matching is performed according to the combination result selected by the user. However, the existing technologies all have the following disadvantages: the accuracy is low, and simple similarity can only be calculated through a single field without considering the association between multiple fields; the long text support effect is poor, and it is difficult to accurately extract the key information of the address from the long text. Summary of the invention

[0003] In view of the above problems, the present invention provides an address similarity calculation method, device, equipment and storage medium, aiming to improve the accuracy of address matching.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for calculating address similarity is provided, the method comprising the following steps:

[0005] Using the BERT model, extracting corpus features from the input multi-field address data, and converting the corpus features into a text feature vector of a structured standard address;

[0006] Taking the structured standard address as a sample, establishing a dynamically updated standard address library;

[0007] According to the importance of different fields in the address and the characteristic performance during the BERT model training process, corresponding weights are set for each field so that the influence of each part can be comprehensively considered in the similarity calculation stage;

[0008] Calculating the similarity between the Chinese text and its corresponding pinyin using the cosine similarity method based on the text feature vector, so as to measure the similarity between addresses in two dimensions;

[0009] Based on the similarity between the Chinese text and its corresponding pinyin, the comprehensive similarity of the multi-field text is determined, and whether two multi-field texts are similar is determined according to a pre-set threshold;

[0010] The standard address library is dynamically updated according to the similarity results between the addresses.

[0011] In some embodiments, before the multi-field address data is input into the BERT model, data preprocessing is performed first, including any one or more of removing noise characters, unifying the format, and filling in missing values.

[0012] In some embodiments, corpus features are constructed through a BERT model, and the corpus features include address keywords, address types, and address relationship information. The BERT model extracts semantic features of address keywords, address types, and address relationship information respectively, performs splicing and fusion, and finally obtains a text feature vector through the fully connected layer of the BERT model.

[0013] In some embodiments, the similarity between the Chinese text and its corresponding pinyin is calculated based on the text feature vector using the cosine similarity method, wherein the similarity of the Chinese text is calculated by using the cosine similarity to calculate the cosine value between the angles of two text feature vectors, and the similarity of the Chinese text and the corresponding pinyin is calculated by converting the text in the text feature vector into pinyin and then calculating the cosine value between the angles of the text feature vectors.

[0014] In some embodiments, the comprehensive similarity of the multi-field text is an average value of the Chinese text similarity and the corresponding pinyin similarity.

[0015] In some embodiments, the standard address library is dynamically updated based on similarity results between addresses. When it is determined that multi-field address data is not similar to any address in the standard address library, it is considered to be a new standard address and added to the standard address library.

[0016] According to a second aspect of an embodiment of the present disclosure, there is provided an address similarity calculation device, the device comprising:

[0017] A text feature vector generation module, used to extract corpus features from the input multi-field address data using the BERT model, and convert the corpus features into a text feature vector of a structured standard address;

[0018] A standard address library establishment module, used to establish a dynamically updated standard address library using the structured standard address as a sample;

[0019] The field weight determination module is used to set corresponding weights for each field according to the importance of different fields in the address and the characteristic performance during the BERT model training process, so as to comprehensively consider the influence of each part in the similarity calculation stage;

[0020] A cosine similarity calculation module, used to calculate the similarity between the Chinese text and its corresponding pinyin according to the text feature vector using the cosine similarity method, and used to measure the similarity between addresses in two dimensions;

[0021] A similarity determination module is used to determine the comprehensive similarity of multi-field texts based on the similarity of Chinese texts and their corresponding pinyin, and to determine whether two multi-field texts are similar according to a preset threshold;

[0022] The standard address library updating module is used to dynamically update the standard address library according to the similarity results between addresses.

[0023] In some embodiments, the device also includes a data cleaning module for performing data preprocessing before the multi-field address data is input into the BERT model, including any one or more of removing noise characters, unifying the format, and filling in missing values.

[0024] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned address similarity calculation method when executing the program.

[0025] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the above-mentioned address similarity calculation method are implemented.

[0026] The embodiments of the present disclosure provide an address similarity calculation method, device and storage medium to solve the problem of low success rate of multiple field matching in existing address matching technology when multiple fields have valid addresses or multiple fields need to be integrated to confirm valid addresses; and the problem of low accuracy of extracting valid addresses from long texts. The method of the present invention uses a BERT model to extract corpus features from input multi-field address data and convert the corpus features into text feature vectors of structured standard addresses; using the structured standard addresses as samples, a dynamically updated standard address library is established; according to the importance of different fields in the address and the characteristics of the BERT model during training, the corpus features are converted into text feature vectors of structured standard addresses; The method is characterized in that a corresponding weight is set for each field so as to comprehensively consider the influence of each part in the similarity calculation stage; the similarity of the Chinese text and its corresponding pinyin is calculated by using the cosine similarity method according to the text feature vector, so as to measure the similarity between addresses in two dimensions; based on the similarity of the Chinese text and its corresponding pinyin, the comprehensive similarity of the multi-field text is determined, and whether two multi-field texts are similar is judged according to a preset threshold; the standard address library is dynamically updated according to the similarity results between the addresses, which has the beneficial effect of not only significantly improving the success rate in the face of multi-field address matching tasks, but also effectively solving the problem of low accuracy in extracting effective addresses in long texts.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present invention and, together with the description, serve to explain the principles of the present invention.

[0029] Figure 1 is a flow chart of a method for calculating address similarity in an embodiment of the present invention;

[0030] Figure 2 is an example diagram of a method for calculating address similarity in an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of the structure of an address similarity calculation device in an embodiment of the present invention;

[0032] Figure 4 It is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0034] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0035] The present invention provides the following embodiments for a method, device and storage medium for calculating address similarity:

[0036] Example 1 is used to illustrate the method for calculating address similarity. Figure 1 , is a flowchart of the address similarity calculation method, and the specific steps include:

[0037] S1. Using the BERT model, extract corpus features from the input multi-field address data and convert the corpus features into text feature vectors of structured standard addresses;

[0038] S2. Using structured standard addresses as samples, establish a dynamically updated standard address library;

[0039] Specifically, a dynamically updated standard address library is established and maintained, and structured standard addresses and those detected and deduplicated are used as samples to establish the standard address library;

[0040] S3. According to the importance of different fields in the address and the characteristic performance during the BERT model training process, corresponding weights are set for each field so as to comprehensively consider the influence of each part in the similarity calculation stage;

[0041] S4. Calculate the similarity between the Chinese text and its corresponding pinyin using the cosine similarity method according to the text feature vector, so as to measure the similarity between the addresses in two dimensions;

[0042] S5, based on the similarity of the Chinese text and its corresponding pinyin, determine the comprehensive similarity of the multi-field text, and judge whether two multi-field texts are similar according to a preset threshold;

[0043] S6. Dynamically update the standard address library based on the similarity results between addresses. Specifically, determine whether the new address is similar to the addresses in the address library based on the multi-field text similarity. If not, add the new address to the address library for updating.

[0044] In some embodiments, before S1, that is, before the multi-field address data is input into the BERT model, data preprocessing is performed, including any one or more of removing noise characters, unifying the format, and filling missing values, so as to ensure the quality of data for subsequent processing.

[0045] In some embodiments, the BERT model is used in S1 to accurately extract possible address entities from the cleaned data and convert them into text feature vectors in the form of structured standard addresses.

[0046] Corpus features are constructed through the BERT model. The corpus features include address keywords, address types, and address relationship information. The BERT model extracts the semantic features of address keywords, address types, and address relationship information respectively, concatenates and fuses them, and finally obtains the text feature vector through the fully connected layer of the BERT model.

[0047] In the specific implementation process, corpus pre-training is first performed. In the exemplary embodiment, corpus features are constructed through the BERT model. The corpus features include address keywords, address types, and address relationship information. Specifically: address keywords are mainly words related to addresses. The embodiment collects historical data and public address data for sorting. Address types are divided into seven categories, including: administrative divisions, roads, regions, house numbers, building addresses, and other non-address information. The weight of each type is different, the administrative division weight is 0.2, the road is 0.3, the region is 0.3, the house number is 0.1, the building address is 0.1, and the weight of other non-address information is 0. Address relationship information includes direction relationship, distance relationship, and topological relationship. The steps of corpus pre-training are as follows:

[0048] Step 1: Address keyword training, for any sentence , in the data processing stage, the address keyword of X is extracted to obtain a new statement representation: , and then Input into BERT model training, extract the semantic features of the sentence, and after training, the vector representation of the sentence is as follows, where BERT() represents the BERT model:

[0049] ;

[0050] Step 2: Address type training, for any statement , in the data processing stage, the address keyword of X is extracted to obtain a new statement representation: , and then Input into BERT model training to extract the semantic features of the sentence. After training, the vector representation of the sentence is as follows: ;

[0051] Step 3: Address relationship information training: In the data processing stage, the address relationship information of X is extracted to obtain a new sentence representation: , and then Input into BERT model training to extract the semantic features of the sentence. After training, the vector representation of the sentence is as follows: ;

[0052] After the three are trained, the corresponding feature vectors are obtained, and the features of the three are fused together. Through the fusion method of feature splicing, the vector of external feature fusion is obtained, which is expressed as follows: .

[0053] After the corpus pre-training is completed, the BERT model is established. Its input is the data processed in the corpus pre-training, and its related distributed word vector features are obtained. The various corpus feature vectors obtained are concatenated and fused, and the final vector representation is obtained through the fully connected layer. Specifically:

[0054] Step 1: After pre-training based on the BERT model, the address keyword vector A, address type vector B and address relationship information vector C are obtained, and then through the activation function, which is specifically expressed as follows:

[0055] ;

[0056] ;

[0057] ;

[0058] Among them, the hyperbolic tangent function tanh is the activation function of the BERT model, which can be The number is mapped to , and strictly monotonic. W, b are variable parameters of the activation function corresponding to different vectors, w1, W2, W3 represent the weights of A, B, C respectively, and b1, b2, b3 represent the corresponding biases;

[0059] Step 2: All vectors are concatenated together through a fully connected layer, which is expressed as follows: ;

[0060] Step 3: Activate the activation function to get the final output vector: .

[0061] For the output vector of the BERT model, that is, the corpus features including address keywords, address types, and address relationship information, the Softmax classifier is used to classify the final experimental results. The specific process is as follows:

[0062] The Softmax classifier obtains a set of probability values ​​by normalizing the output vector of the BERT model, and classifies the relationship (similar or dissimilar) by judging the size of the probability. Through the Softmax classifier i The conditional probability of this relationship is as follows: ,in, Represents inputting one of the categories. Represents all categories, To assign the correct classification label to the given data The normalized probability of . During the training process, the objective function used is the cross entropy loss function. The calculation is as follows: ,in , They are represented as true value and predicted value respectively, M is the sample, and n is the number of samples.

[0063] The stochastic gradient descent method is used in the model optimization process, and the model parameters are updated by back propagation method. The calculation process is as follows: ,in, Represents the learning rate.

[0064] In S4, the similarity between the Chinese text and its corresponding pinyin is calculated based on the text feature vector using the cosine similarity method, wherein the similarity of the Chinese text is calculated by using the cosine similarity to calculate the cosine value between the angles of two text feature vectors, and the similarity between the Chinese text and the corresponding pinyin is calculated by converting the text in the text feature vector into pinyin and then calculating the cosine value between the angles of the text feature vectors.

[0065] The comprehensive similarity of multi-field text in S5 is the average of the Chinese text similarity and its corresponding pinyin similarity.

[0066] In S6, the standard address library is dynamically updated according to the similarity results between addresses. When it is determined that the multi-field address data is not similar to the addresses in the standard address library, it is considered to be a new standard address and added to the standard address library.

[0067] In an exemplary embodiment of the present invention, Figure 2 As shown, the historical address content is input into the address recognition model, and the output result is used to establish a special address directory for letter objects, that is, a standard address library. For the latest address content, it is input into the address recognition model, and the corpus feature vector is output to the address similarity model. The similarity is compared with the standard address in the special address directory for letter objects to obtain the similarity comparison result, and the special address directory for letter objects is updated with the new standard address.

[0068] Another embodiment is used to illustrate an address similarity calculation device, such as Figure 3 As shown, the device 300 includes:

[0069] A text feature vector generation module 310 is used to extract corpus features from the input multi-field address data using the BERT model, and convert the corpus features into a text feature vector of a structured standard address;

[0070] The standard address library establishment module 320 is used to establish a dynamically updated standard address library using the structured standard address as a sample;

[0071] The field weight determination module 330 is used to set corresponding weights for each field according to the importance of different fields in the address and the characteristic performance during the BERT model training process, so as to comprehensively consider the influence of each part in the similarity calculation stage;

[0072] A cosine similarity calculation module 340, used to calculate the similarity between the Chinese text and its corresponding pinyin according to the text feature vector using the cosine similarity method, and used to measure the similarity between addresses in two dimensions;

[0073] A similarity determination module 350 is used to determine the comprehensive similarity of the multi-field text based on the similarity of the Chinese text and its corresponding pinyin, and to determine whether two multi-field texts are similar according to a preset threshold;

[0074] The standard address library updating module 360 ​​is used to dynamically update the standard address library according to the similarity results between addresses.

[0075] Preferably, if Figure 3 As shown, the device 300 also includes a data cleaning module 370, which is used to perform data preprocessing before the multi-field address data is input into the BERT model, including any one or more of removing noise characters, unifying the format, and filling in missing values.

[0076] In addition to the upper module, the device 300 may also include other components. However, since these components are irrelevant to the content of the embodiment of the present disclosure, their illustration and description are omitted here.

[0077] The other specific working processes of the address similarity calculation device 300 refer to the description of the above-mentioned base address similarity calculation method embodiment, which will not be repeated here.

[0078] Another embodiment is used to illustrate that the device of the present invention can also be used by Figure 4 The architecture of the computing device shown is implemented. Figure 4 The architecture of the computing device is shown. Figure 4As shown, a computer device 410, a device bus 430, one or more CPUs 440, an input / output 420, a memory 450, etc. The memory 450 can store various data or files used for computer processing and / or communication and program instructions including the address similarity calculation method of the embodiment executed by the CPU. Figure 4 The architecture shown is only exemplary and can be adjusted according to actual needs when implementing different devices. Figure 4 One or more components in. The memory 450, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the address similarity calculation method in the embodiment of the present invention (for example, the text feature vector generation module 310, the standard address library establishment module 320, the field weight determination module 330, the cosine similarity calculation module 340, the similarity determination module 350 and the standard address library update module 360 ​​in the address similarity calculation device 300). One or more CPUs 440 execute various functional applications and data processing of the device of the present invention by running the software programs, instructions and modules stored in the memory 450, that is, to implement the above-mentioned address similarity calculation method, which includes the following steps:

[0079] Using the BERT model, extracting corpus features from the input multi-field address data, and converting the corpus features into a text feature vector of a structured standard address;

[0080] Taking the structured standard address as a sample, establishing a dynamically updated standard address library;

[0081] According to the importance of different fields in the address and the characteristic performance during the BERT model training process, corresponding weights are set for each field so that the influence of each part can be comprehensively considered in the similarity calculation stage;

[0082] Calculating the similarity between the Chinese text and its corresponding pinyin using the cosine similarity method based on the text feature vector, so as to measure the similarity between addresses in two dimensions;

[0083] Based on the similarity between the Chinese text and its corresponding pinyin, the comprehensive similarity of the multi-field text is determined, and whether two multi-field texts are similar is determined according to a pre-set threshold;

[0084] The standard address library is dynamically updated according to the similarity results between the addresses.

[0085] Of course, the processor of the server provided in the embodiment of the present invention is not limited to executing the operations of the method described above, but can also execute related operations in the address similarity calculation method provided in any embodiment of the present invention.

[0086] The memory 450 may mainly include a program storage area and a data storage area, wherein the program storage area may store applications required for operating the device and at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 450 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 450 may further include a memory remotely arranged relative to one or more CPUs 440, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0087] The input / output 420 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The input / output 420 may also include a display device such as a display screen.

[0088] The embodiment of the present invention also provides a non-temporary computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, the address similarity calculation method described in the above embodiment is implemented. The computer-readable storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution device, device or device.

[0089] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution device, apparatus, or device.

[0090] The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0091] In addition, other specific working processes of a non-temporary computer-readable storage medium refer to the description of the above-mentioned address similarity calculation method embodiment, which will not be repeated here.

[0092] The address similarity calculation method, device and storage medium provided in the above embodiments are combined to solve the problem of low success rate of multiple field matching in existing address matching technology when multiple fields have valid addresses or multiple fields need to be integrated to confirm the valid address; and the problem of low accuracy of extracting valid addresses from long texts. The method of the present invention uses the BERT model to extract corpus features from the input multi-field address data and convert the corpus features into text feature vectors of structured standard addresses; using the structured standard addresses as samples, a dynamically updated standard address library is established; according to the importance of different fields in the address and the features in the BERT model training process, the corpus features are converted into text feature vectors of structured standard addresses. performance, setting corresponding weights for each field so as to comprehensively consider the influence of each part in the similarity calculation stage; calculating the similarity of the Chinese text and its corresponding pinyin using the cosine similarity method according to the text feature vector, which is used to measure the similarity between addresses in two dimensions; determining the comprehensive similarity of the multi-field text based on the similarity of the Chinese text and its corresponding pinyin, and judging whether two multi-field texts are similar according to a pre-set threshold; dynamically updating the standard address library according to the similarity results between the addresses, which has the beneficial effect of not only significantly improving the success rate in the face of multi-field address matching tasks, but also effectively solving the problem of low accuracy in extracting valid addresses in long texts.

[0093] In this document, the terms "comprises," "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such step or method.

[0094] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for calculating address similarity, characterized in that: The method comprises the following steps: Using the BERT model, extracting corpus features from the input multi-field address data, and converting the corpus features into a text feature vector of a structured standard address; Taking the structured standard address as a sample, establishing a dynamically updated standard address library; According to the importance of different fields in the address and the characteristic performance during the BERT model training process, set corresponding weights for each field; Calculating the similarity between the Chinese text and its corresponding pinyin using the cosine similarity method according to the text feature vector; Based on the similarity between the Chinese text and its corresponding pinyin, the comprehensive similarity of the multi-field text is determined, and whether two multi-field texts are similar is determined according to a pre-set threshold; Dynamically update the standard address library according to the similarity results between addresses; Corpus features are constructed through the BERT model, and the corpus features include address keywords, address types, and address relationship information. The BERT model extracts semantic features of address keywords, address types, and address relationship information respectively, performs splicing and fusion, and finally obtains a text feature vector through the fully connected layer of the BERT model.

2. The method for calculating address similarity according to claim 1, characterized in that: Before the multi-field address data is input into the BERT model, data preprocessing is performed first, including removing any one or more of noise characters, unifying the format, and filling missing values.

3. The method for calculating address similarity according to claim 1, characterized in that: The similarity between the Chinese text and its corresponding pinyin is calculated based on the text feature vector using the cosine similarity method, wherein the similarity of the Chinese text is calculated by using the cosine similarity to calculate the cosine value between the angles of two text feature vectors, and the similarity of the Chinese text and the corresponding pinyin is calculated by converting the text in the text feature vector into pinyin and then calculating the cosine value between the angles of the text feature vectors.

4. The method for calculating address similarity according to claim 1, characterized in that: The comprehensive similarity of the multi-field text is the average value of the Chinese text similarity and the corresponding pinyin similarity.

5. The method for calculating address similarity according to claim 1, characterized in that: The standard address library is dynamically updated according to the similarity results between addresses. When it is determined that the multi-field address data is not similar to any address in the standard address library, it is considered to be a new standard address and added to the standard address library.

6. An address similarity calculation device, characterized in that: The device comprises: A text feature vector generation module, used to extract corpus features from the input multi-field address data using the BERT model, and convert the corpus features into a text feature vector of a structured standard address; A standard address library establishment module, used to establish a dynamically updated standard address library using the structured standard address as a sample; The field weight determination module is used to set corresponding weights for each field based on the importance of different fields in the address and the characteristic performance during the BERT model training process; A cosine similarity calculation module, used to calculate the similarity between the Chinese text and its corresponding pinyin according to the text feature vector using the cosine similarity method; A similarity determination module is used to determine the comprehensive similarity of multi-field texts based on the similarity of Chinese texts and their corresponding pinyin, and to determine whether two multi-field texts are similar according to a preset threshold; A standard address library updating module, used for dynamically updating the standard address library according to the similarity results between addresses; In the text feature vector generation module, corpus features are constructed through the BERT model, and the corpus features include address keywords, address types and address relationship information. The BERT model extracts the semantic features of address keywords, address types and address relationship information respectively, performs splicing and fusion, and finally obtains the text feature vector through the BERT model fully connected layer.

7. The address similarity calculation device according to claim 6, characterized in that: The device also includes a data cleaning module for performing data preprocessing before the multi-field address data is input into the BERT model, including any one or more of removing noise characters, unifying the format, and filling in missing values.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the address similarity calculation method according to any one of claims 1 to 5 are implemented.

9. A non-transitory computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the address similarity calculation method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Field matching method and terminal equipment

    CN108629046A

  • Address standardization processing method, device, equipment and computer readable storage medium

    CN110895651A

  • Address matching method and device, electronic equipment and storage medium

    CN112818685A

  • Address matching method and device based on deep learning, equipment and storage medium

    CN117473025A

  • Electricity utilization address similarity matching method based on semantics

    CN117494711A