High-precision address data processing method and system based on named entity recognition

Abstract

The invention relates to the technical field of data processing, in particular to a high-precision address data processing method and system based on named entity recognition. A high-precision address data processing method based on named entity recognition comprises the following steps: S1, preprocessing address data to obtain an address matching rule base; s2, setting a first-level address matching rule and a second-level address matching rule based on the address matching rule base; s3, acquiring address information of an alarm person, and matching the address information of the alarm person with a secondary address matching rule; and S4, obtaining a first-level address matching rule corresponding to the alarm personnel based on a matching result. By analyzing the address data of the alarm personnel, the first-level address matching rule matched with the address data of the alarm personnel is obtained, the historical alarm occurrence address of the alarm personnel or historical alarm personnel is analyzed to adjust the second-level address matching rule, and the matching speed and accuracy of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a high-precision address data processing method and system based on named entity recognition. Background Art

[0002] With the rapid development of the Internet and big data technology, the collection of police data has become increasingly important. Police data has not only gradually become a key indicator of regional public security, but also can be used to analyze public security trends and support police stations to better carry out security prevention work, thereby focusing on analyzing the high-incidence locations and time patterns of high-incidence police incidents. The purpose of address data matching is to compare and associate the address information entered by the user with the information in the standard address database, so as to achieve standardization and accurate positioning of the address. However, the existing police data collection methods face problems such as diversified crime addresses, long matching time, and huge resource consumption. Summary of the invention

[0003] In order to overcome the shortcomings of inaccurate address data processing and high resource consumption, the present invention provides a high-precision address data processing method and system based on named entity recognition.

[0004] The technical solution of the present invention is: a high-precision address data processing method based on named entity recognition, comprising the following steps:

[0005] S1: After preprocessing the address data, an address matching rule base is obtained;

[0006] S2: Setting a primary address matching rule and a secondary address matching rule based on the address matching rule library;

[0007] S3: Acquire the address information of the alarm person, and match the address information of the alarm person with the secondary address matching rule;

[0008] S4: Obtaining the primary address matching rule corresponding to the alarm personnel based on the matching result.

[0009] Preferably, after the address data is preprocessed, an address matching rule base is obtained, including: preprocessing the address data to remove abnormal data, fill missing values, and remove duplicate data, and processing the address data using symbols to obtain the address matching rule base.

[0010] Preferably, the setting of the first-level address matching rules and the second-level address matching rules based on the address matching rule base includes: setting a first preset number of first-level address matching rules according to the address planning data and the address matching rule base, and making the second-level address matching rules correspond to the first-level address matching rules, wherein a second preset number of address matching rule bases are used for combination to obtain the second-level address matching rules.

[0011] Preferably, the acquiring the address information of the alarming person and matching the address information of the alarming person with the secondary address matching rule comprises: matching the address information of the alarming person with a plurality of secondary address matching rules;

[0012] If the first part of the address rule in the secondary address matching rule does not successfully match the first part of the address information of the alarm person, then continue to use the secondary address matching rule to match the alarm person;

[0013] If the first part of the address rule in the secondary address matching rule successfully matches the first part of the address information of the alarm person, the subordinate address rule of the first part of the address rule in the secondary address matching rule is obtained and used as the second part of the address rule, and sorted based on the second part of the address rule.

[0014] Preferably, the sorting based on the second part of the address rule includes: obtaining the second part of the address rule, obtaining the historical alarm personnel within the preset time period of all the second part of the address rules in the secondary address matching rule, and obtaining the historical alarm occurrence addresses of the historical alarm personnel based on the historical alarm personnel, and matching the alarm personnel with the historical alarm personnel;

[0015] If there is an alarm person among the historical alarm persons, the historical alarm occurrence addresses of the historical alarm person are sorted using the LFU algorithm;

[0016] If there is no alarm person among the historical alarm persons, the historical alarm occurrence addresses of all historical alarm persons are sorted using the LFU algorithm.

[0017] Preferably, if there is an alarm person among the historical alarm persons, the historical alarm addresses of the alarm person are sorted using the LFU algorithm, including: sorting the historical alarm addresses of the alarm persons based on the LFU algorithm to obtain a sequence of historical alarm addresses; using named entity recognition technology to obtain the information submission adjustment number of each historical alarm address, and setting the length of the historical alarm address sequence based on the number of information submission types of each historical alarm address of the alarm person.

[0018] Preferably, if there is an alarm person among the historical alarm persons, the historical alarm occurrence addresses of the alarm persons are sorted using the LFU algorithm, including: using the named entity recognition technology on the sorted historical alarm occurrence address sequence to obtain the information submission adjustment number of the alarm occurrence address, and comparing and analyzing the data of the alarm occurrence address with the most repetitions with the data of the alarm occurrence address after the most recent replacement time;

[0019] If the number of information submission adjustments for the alarm occurrence address with the most repetitions is the same as the number of information submission adjustments for the alarm occurrence address after the most recent replacement time, the second part of the address rules are sorted according to the number of information submission adjustments for the alarm occurrence address;

[0020] If the information submission adjustment number of the alarm occurrence address with the most repetitions is different from the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the information submission adjustment number of the alarm occurrence address is selected to obtain the final information submission adjustment number of the alarm occurrence address, and the second part of the address rules are sorted according to the final information submission adjustment number of the alarm occurrence address.

[0021] Preferably, if the information submission adjustment number of the alarm occurrence address with the most repetitions is the same as the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the second part of the address rules are sorted according to the information submission adjustment number of the alarm occurrence address, including:

[0022] When each second-part address rule has a subordinate address rule, the second-part address rule with the subordinate address matching rule is processed in advance;

[0023] When there is no subordinate address rule in each second part address rule, the alarm personnel are sorted according to the time sequence in which they appear in each second part address rule;

[0024] Repeat the analysis until the N-level address rules are reached, and construct the second-level address matching rule sequence.

[0025] Preferably, if the information submission adjustment number of the alarm occurrence address with the most repeated times is different from the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the information submission adjustment number of the alarm occurrence address is selected to obtain the final information submission adjustment number of the alarm occurrence address, and the secondary address matching rules are sorted according to the final information submission adjustment number of the alarm occurrence address, including: obtaining the total number of the same information submission adjustment numbers of each information in the historical alarm occurrence address sequence, taking the information submission adjustment number of the alarm occurrence address with the most repeated times as the final information submission adjustment number of the alarm occurrence address, and sorting the secondary address matching rules to obtain the secondary address matching rule order.

[0026] Preferably, a high-precision address data processing system based on named entity recognition comprises:

[0027] An address data processing module is used to pre-process the address data by removing abnormal data, filling missing values, and removing duplicate data, and to process the address data using symbols to obtain an address matching rule library;

[0028] An address rule processing module, used to set a primary address matching rule and a secondary address matching rule according to the address planning data and the address matching rule library;

[0029] Address rule corresponding module, used for matching the secondary address matching rule with the primary address matching rule;

[0030] An address division module is used to divide the secondary address matching rules or the address information of the alarm personnel into the first part of the address rules, the second part of the address rules and the Nth level of partial address rules;

[0031] An address matching module is used to match the first part of the address rules in the secondary address matching rules with the first part of the address information of the alarm personnel;

[0032] The address sorting module is used to sort the secondary address matching rules according to the LFU algorithm and the historical alarm occurrence address data of the historical alarm personnel.

[0033] Beneficial effects: The present invention uses the method of connectors to quickly and intelligently identify the location of the alarm, reduce the amount of data storage, adjust the secondary address matching rules by analyzing the addresses of historical alarms of each alarm personnel or historical alarm personnel, sort out a reasonable sorting order when the number of rules is huge, reduce the amount of data calculation, and use the LFU algorithm to analyze the writing habits of the alarm personnel, combine the number of times the alarm personnel writes and the time relationship to reduce the number of rule matches and speed up the matching speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a high-precision address data processing method based on named entity recognition according to the present invention;

[0035] Figure 2 A block diagram of the address rule setting module of the present invention. DETAILED DESCRIPTION

[0036] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention.

[0037] Embodiment 1: A high-precision address data processing method based on named entity recognition, such as Figure 1-2 As shown, the following steps are included:

[0038] S1: After preprocessing the address data, an address matching rule base is obtained;

[0039] The address data is preprocessed to remove abnormal data, fill missing values, and remove duplicate data, and the address data is processed using symbols to obtain an address matching rule library.

[0040] It should be noted that complete and correct data is obtained by processing address data, and specific address matching rule libraries for each region, such as a name library, a surrounding library, and an extension library, are obtained by using connectors or wildcards.

[0041] S2: Setting a primary address matching rule and a secondary address matching rule based on the address matching rule library;

[0042] According to the address planning data and the address matching rule library, a first preset number of first-level address matching rules are set, and the second-level address matching rules are matched with the first-level address matching rules, wherein a second preset number of address matching rule libraries are used for combination to obtain the second-level address matching rules.

[0043] It should be noted that the first-level address matching rules of three-level division are obtained through the address planning data and the address matching rule library. For example, for the A Hotel B Block Store 1105, it can be divided into the first-level address matching rules of residence-hotel-hotel, and the second-level address matching rules corresponding to the first-level address matching rules are: name 04-main extension 01-inside 03 or name 04-inside 03 or name 04-main extension 01-inside 01 or name 04-inside 01.

[0044] S3: Acquire the address information of the alarm person, and match the address information of the alarm person with the secondary address matching rule;

[0045] Match the address information of the alarm personnel with multiple secondary address matching rules;

[0046] If the first part of the address rule in the secondary address matching rule does not successfully match the first part of the address information of the alarm person, then continue to use the secondary address matching rule to match the alarm person;

[0047] If the first part of the address rule in the secondary address matching rule successfully matches the first part of the address information of the alarm person, the subordinate address rule of the first part of the address rule in the secondary address matching rule is obtained and used as the second part of the address rule, and sorted based on the second part of the address rule.

[0048] It should be noted that the address information of the alarming person is matched with multiple secondary address matching rules, for example: the secondary address matching rules are: Name 04-Internal 03-Main extension 01 or Name 04-Internal 03 or Name 04-Internal 01-Main extension 01 or Name 04-Internal 01; the address data of the alarming person is: A Hotel B Block Store 1105, where Name 04 is the first part of the address rule of the secondary address matching rules, and A Hotel is the first part of the address rule for the alarming person. If they are not the same, the remaining secondary address matching rules will continue to be used to match the alarming person. If they are the same, the second part of the address rules will be analyzed, where Internal 03 or Internal 01 is the second part of the address rule of the secondary address matching rules, and B Block Store is the second part of the address rule for the alarming person.

[0049] Obtain the second part of the address rules, obtain the historical alarm personnel within the preset time period of all the second part of the address rules in the secondary address matching rules, and obtain the historical alarm occurrence addresses of the historical alarm personnel based on the historical alarm personnel, and match the alarm personnel with the historical alarm personnel;

[0050] If there is an alarm person among the historical alarm persons, the historical alarm occurrence addresses of the historical alarm person are sorted using the LFU algorithm;

[0051] If there is no alarm person among the historical alarm persons, the historical alarm occurrence addresses of all historical alarm persons are sorted using the LFU algorithm.

[0052] It should be noted that, when obtaining historical alarm personnel within a preset time period in internal 03 or internal 01, such as historical alarm personnel D and historical alarm personnel E, if historical alarm personnel D is the same as the alarm personnel, then the LFU algorithm is used to sort them according to the historical alarm occurrence address of historical alarm personnel D; if it does not exist, then all historical alarm occurrence addresses of historical alarm personnel D and historical alarm personnel E are sorted using the LFU algorithm.

[0053] Based on the LFU algorithm, the historical alarm addresses of the alarm personnel are sorted to obtain the historical alarm address sequence; the named entity recognition technology is used to obtain the information submission adjustment number of each historical alarm address, and the length of the historical alarm address sequence is set based on the number of information submission types of each historical alarm address of the alarm personnel.

[0054] It should be noted that the named entity recognition technology is used to obtain the information submission adjustment number of each historical alarm occurrence address. For example: in name 04-internal 03-main extension 01 or name 04-internal 03 or name 04-internal 01-main extension 01 or name 04-internal 01, the information submission adjustment number of name 04-internal 03-main extension 01 is 3, and the number of information submission types is 4, then the length of the historical alarm occurrence address sequence obtained using LFU is 4.

[0055] Use named entity recognition technology on the sorted historical alarm address sequence to obtain the number of information submission adjustments for the alarm address, and compare and analyze the data of the alarm address with the most repetitions with the data of the alarm address after the most recent replacement time;

[0056] If the number of information submission adjustments for the alarm occurrence address with the most repetitions is the same as the number of information submission adjustments for the alarm occurrence address after the most recent replacement time, the second part of the address rules are sorted according to the number of information submission adjustments for the alarm occurrence address;

[0057] If the information submission adjustment number of the alarm occurrence address with the most repetitions is different from the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the information submission adjustment number of the alarm occurrence address is selected to obtain the final information submission adjustment number of the alarm occurrence address, and the second part of the address rules are sorted according to the final information submission adjustment number of the alarm occurrence address.

[0058] When each second-part address rule has a subordinate address rule, the second-part address rule with the subordinate address matching rule is processed in advance;

[0059] When there is no subordinate address rule in each second part address rule, the alarm personnel are sorted according to the time sequence in which they appear in each second part address rule;

[0060] Repeat the analysis until the N-level address rules are reached, and construct the second-level address matching rule sequence.

[0061] It should be noted that the named entity recognition technology is used for the sorted historical alarm address sequence to obtain the information submission adjustment number of the alarm address, for example: the information submission adjustment number in the sequence is 4, 3, 3; the time order is 1, 3, 2, where 1 is the earliest entry and 3 is the latest entry into the sequence; the number of repetitions of the alarm address is 1, 10, 1, and the information submission adjustment number of the alarm address with the largest number of repetitions is 3. When the information submission adjustment number of the alarm address with the largest number of repetitions is the same as the information submission adjustment number of the alarm address after the most recent replacement time, both of which are 3, it is determined that the person's writing habit is to write three-level partial address rules, and whether there is a third part of the address rule, that is, the main extension 01, if it exists, it is advanced so that the secondary address matching rule with the three-level partial address rule can be traversed as soon as possible. If it does not exist, it is sorted according to the time order of the alarm personnel appearing in each second part of the address rules, and the analysis is repeated until the N-level address rules are reached to construct the secondary address matching rule sequence.

[0062] Obtain the total number of identical information submission adjustment numbers in the historical alarm occurrence address sequence, take the information submission adjustment number of the alarm occurrence address with the most repetitions as the final information submission adjustment number of the alarm occurrence address, and sort the secondary address matching rules to obtain the secondary address matching rule order.

[0063] It should be noted that, after selecting the adjustment number of the alarm occurrence address information submitted, the final adjustment number of the alarm occurrence address information submitted is obtained, and the secondary address matching rules are sorted according to the final adjustment number of the alarm occurrence address information submitted. For example: if the adjustment number of the information submitted in the sequence is 4, 3, 3, 2; the time order is 1, 3, 2, 4; the number of repetitions of the alarm occurrence address is 9, 10, 1, 1, the number of repetitions of the information submission adjustment number 4 is 9, and the number of repetitions of the information submission adjustment number 3 is 10+1=11, then the information submission adjustment number 3 is used as the final adjustment number of the alarm occurrence address information submitted, and the secondary address matching rules are sorted to obtain the order of the secondary address matching rules.

[0064] S4: Obtaining the primary address matching rule corresponding to the alarm personnel based on the matching result.

[0065] Embodiment 2: Based on Embodiment 1, a high-precision address data processing system based on named entity recognition includes:

[0066] An address data processing module is used to pre-process the address data by removing abnormal data, filling missing values, and removing duplicate data, and to process the address data using symbols to obtain an address matching rule library;

[0067] An address rule processing module, used to set a primary address matching rule and a secondary address matching rule according to the address planning data and the address matching rule library;

[0068] Address rule corresponding module, used for matching the secondary address matching rule with the primary address matching rule;

[0069] An address division module is used to divide the secondary address matching rules or the address information of the alarm personnel into the first part of the address rules, the second part of the address rules and the Nth level of partial address rules;

[0070] An address matching module is used to match the first part of the address rules in the secondary address matching rules with the first part of the address information of the alarm personnel;

[0071] The address sorting module is used to sort the secondary address matching rules according to the LFU algorithm and the historical alarm occurrence address data of the historical alarm personnel.

[0072] Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solutions obtained by equivalent replacement or equivalent transformation shall fall within the protection scope of the present invention.

Claims

1. A high-precision address data processing method based on named entity recognition, characterized in that: The following steps are involved: S1: After preprocessing the address data, an address matching rule base is obtained; S2: Setting a primary address matching rule and a secondary address matching rule based on the address matching rule library; S3: Acquire the address information of the alarm person, and match the address information of the alarm person with the secondary address matching rule; S4: Obtain the primary address matching rule corresponding to the alarm personnel based on the matching result.

2. A high-precision address data processing method based on named entity recognition as claimed in claim 1, characterized in that: After the address data is preprocessed, an address matching rule base is obtained, including: preprocessing the address data to remove abnormal data, fill missing values, and remove duplicate data, and using symbols to process the address data to obtain the address matching rule base.

3. A high-precision address data processing method based on named entity recognition as claimed in claim 1, characterized in that: The method of setting the first-level address matching rules and the second-level address matching rules based on the address matching rule library includes: setting a first preset number of first-level address matching rules according to address planning data and the address matching rule library, and making the second-level address matching rules correspond to the first-level address matching rules, wherein a second preset number of address matching rule libraries are used for combination to obtain the second-level address matching rules.

4. A high-precision address data processing method based on named entity recognition as claimed in claim 1, characterized in that: The obtaining of the address information of the alarm personnel and matching the address information of the alarm personnel with the secondary address matching rules includes: matching the address information of the alarm personnel with a plurality of secondary address matching rules; If the first part of the address rule in the secondary address matching rule does not successfully match the first part of the address information of the alarm person, then continue to use the secondary address matching rule to match the alarm person; If the first part of the address rule in the secondary address matching rule successfully matches the first part of the address information of the alarm person, the subordinate address rule of the first part of the address rule in the secondary address matching rule is obtained and used as the second part of the address rule, and sorted based on the second part of the address rule.

5. A high-precision address data processing method based on named entity recognition as claimed in claim 4, characterized in that: The sorting based on the second part of the address rule includes: obtaining the second part of the address rule, obtaining the historical alarm personnel within the preset time period of all the second part of the address rules in the secondary address matching rule, and obtaining the historical alarm occurrence address of the historical alarm personnel based on the historical alarm personnel, and matching the alarm personnel with the historical alarm personnel; If there is an alarm person among the historical alarm persons, the historical alarm occurrence addresses of the historical alarm person are sorted using the LFU algorithm; If there is no alarm person among the historical alarm persons, the historical alarm occurrence addresses of all historical alarm persons are sorted using the LFU algorithm.

6. A high-precision address data processing method based on named entity recognition as claimed in claim 5, characterized in that: If there is an alarm person among the historical alarm persons, the historical alarm addresses of the alarm person are sorted using the LFU algorithm, including: sorting the historical alarm addresses of the alarm persons based on the LFU algorithm to obtain a sequence of historical alarm addresses; using named entity recognition technology to obtain the information submission adjustment number of each historical alarm address, and setting the length of the historical alarm address sequence based on the number of information submission types of each historical alarm address of the alarm person.

7. A high-precision address data processing method based on named entity recognition as claimed in claim 5, characterized in that: If there is an alarm person among the historical alarm persons, the historical alarm occurrence addresses of the alarm persons are sorted using the LFU algorithm, including: using the named entity recognition technology on the sorted historical alarm occurrence address sequence to obtain the information submission adjustment number of the alarm occurrence address, and comparing and analyzing the data of the alarm occurrence address with the most repeated times with the data of the alarm occurrence address after the most recent replacement time; If the number of information submission adjustments for the alarm occurrence address with the most repetitions is the same as the number of information submission adjustments for the alarm occurrence address after the most recent replacement time, the second part of the address rules are sorted according to the number of information submission adjustments for the alarm occurrence address; If the information submission adjustment number of the alarm occurrence address with the most repetitions is different from the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the information submission adjustment number of the alarm occurrence address is selected to obtain the final information submission adjustment number of the alarm occurrence address, and the second part of the address rules are sorted according to the final information submission adjustment number of the alarm occurrence address.

8. A high-precision address data processing method based on named entity recognition as claimed in claim 7, characterized in that: If the information submission adjustment number of the alarm occurrence address with the most repetitions is the same as the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the second part of the address rules are sorted according to the information submission adjustment number of the alarm occurrence address, including: When each second-part address rule has a subordinate address rule, the second-part address rule with the subordinate address matching rule is processed in advance; When each second-part address rule does not have a subordinate address rule, the alarm personnel are sorted according to the time sequence in which they appear in each second-part address rule; Repeat the analysis until the N-level address rules are reached, and construct the second-level address matching rule sequence.

9. A high-precision address data processing method based on named entity recognition as claimed in claim 7, characterized in that: If the information submission adjustment number of the alarm occurrence address with the most repetitions is different from the information submission adjustment number of the alarm occurrence address after the most recent replacement time, the information submission adjustment number of the alarm occurrence address is selected to obtain the final information submission adjustment number of the alarm occurrence address, and the secondary address matching rules are sorted according to the final information submission adjustment number of the alarm occurrence address, including: Obtain the total number of identical information submission adjustment numbers in the historical alarm occurrence address sequence, take the information submission adjustment number of the alarm occurrence address with the most repetitions as the final information submission adjustment number of the alarm occurrence address, and sort the secondary address matching rules to obtain the secondary address matching rule order.

10. A high-precision address data processing system based on named entity recognition, characterized in that: include: An address data processing module is used to pre-process the address data by removing abnormal data, filling missing values, and removing duplicate data, and to process the address data using symbols to obtain an address matching rule library; An address rule processing module, used to set a primary address matching rule and a secondary address matching rule according to the address planning data and the address matching rule library; Address rule corresponding module, used for matching the secondary address matching rule with the primary address matching rule; An address division module is used to divide the secondary address matching rules or the address information of the alarm personnel into the first part of the address rules, the second part of the address rules and the Nth level of partial address rules; An address matching module is used to match the first part of the address rules in the secondary address matching rules with the first part of the address information of the alarm personnel; The address sorting module is used to sort the secondary address matching rules according to the LFU algorithm and the historical alarm occurrence address data of the historical alarm personnel.

Images