Address extraction method for public opinion text
By performing supervised fine-tuning and few-sample prompts on the pre-trained large model, combined with address decomposition and orientation analysis, the problems of high noise, colloquialism and vague descriptions in public opinion texts are solved, and high-accuracy extraction and precise coordinate positioning of address information in public opinion texts are achieved.
Patent Information
- Application Number
- CN202510587498.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-09-12
AI Technical Summary
When processing public opinion texts, existing technologies face problems such as high noise, colloquialisms, and vague descriptions, resulting in low accuracy in address extraction, lack of contextual understanding capabilities, and a systematic offset calculation process.
A pre-trained large model combined with supervised fine-tuning and few-sample prompting methods is used to extract address information from public opinion texts, and the coordinates of the target address are calculated through address decomposition and direction analysis.
The accuracy and robustness of address recognition are improved, and the target address can be accurately located, which solves the shortcomings of traditional methods in processing additional location information and achieving fine positioning.
Smart Images
Figure CN120632084A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of address extraction from public opinion texts, and in particular to an address extraction method for public opinion texts. Background Art
[0002] In the field of address entity extraction, early work often used methods based on regular expressions and keyword matching to detect potential address elements in text. These methods relied on predefined rules or fixed keyword lists, effectively identifying some typical geographic designators. However, when faced with colloquial, dialectal, nicknamed, and ambiguous address descriptions, rule- or keyword-based solutions often struggled with the complex and varied expressions, resulting in limited applicability and accuracy.
[0003] With the continuous advancement of natural language processing technology, methods for extracting address entities based on named entity recognition (NER) have emerged. These methods typically combine statistical learning models with feature engineering and sequence annotation of text sequences. Compared to pure rule matching, traditional NER methods can capture more contextual information, improving the recall and accuracy of address entities. However, when dealing with highly diverse and noisy public opinion text, these methods still rely on large amounts of annotated data and rich feature design, resulting in limited generalization capabilities.
[0004] The rapid development of large language models has brought new solutions to address entity extraction. Compared with traditional NER methods, large language models are more adaptable to scenarios such as non-standard text, colloquial descriptions, and typos. However, directly using large models often leads to suboptimal output results, with issues such as insufficient accuracy and robustness.
[0005] With respect to the above-mentioned prior art, the processing method for extracting geographic address information from text and locating it faces the following problems:
[0006] 1. Public opinion texts are noisy, colloquial, and contain ambiguous descriptions: Current rule-based or traditional NER methods are often affected by text noise, colloquial expressions, and ambiguous descriptions when processing public opinion texts, resulting in low accuracy in address extraction.
[0007] 2. Lack of contextual understanding: Existing methods find it difficult to automatically correct and complete typos, abbreviated expressions, or incomplete information in texts using contextual semantics, thus affecting the robustness of address recognition.
[0008] 3. Lack of a systematic offset calculation process: For text containing direction and distance descriptions such as "200 meters north", existing technologies generally lack a process for further offset calculation of the extracted address to achieve precise coordinate positioning, resulting in the inability to obtain a more accurate target location from the base address. Summary of the Invention
[0009] The purpose of the present invention is to provide an address extraction method for public opinion texts to address the above-mentioned problems. The method is to automatically extract address information from public opinion texts, which can identify address elements such as streets, communities, roads, subway stations, and unit names contained in the texts and convert them into usable coordinates to provide data support for subsequent public opinion hotspot analysis.
[0010] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is as follows:
[0011] According to one aspect of the present invention, a method for extracting addresses from public opinion text is provided, comprising the following steps:
[0012] S1. Data acquisition: crawling the public opinion text data of the address to be extracted from the public opinion platform;
[0013] S2. Large model address extraction: Input the public opinion text data into the pre-trained large model that has been optimized through supervised fine-tuning and few-shot prompts, and use the pre-trained large model to extract the address information of the public opinion text data;
[0014] S3. Address decomposition and orientation analysis: Perform structured decomposition on the address information extracted from the large model to obtain the base address and orientation description;
[0015] S4. Address coordinate calculation: perform coordinate calculation based on the reference address and orientation description to obtain the target address coordinates;
[0016] S5. Verification: Compare the calculated coordinates of the final target address with the true coordinates, and calculate the offset error and average error.
[0017] Preferably, in step S1, the public opinion text data is text information of user complaints and feedback.
[0018] Preferably, in step S2, extracting address information from public opinion text data includes the following steps:
[0019] Model supervised fine-tuning: Based on the pre-trained large model, supervised fine-tuning is performed using high-quality labeled data for address extraction tasks;
[0020] Few-sample prompts: Design a small set of high-quality example prompts for community names, intersections, unit names, and expressions without addresses to guide the pre-trained large model to output standardized address information during the inference phase, thereby obtaining an optimized pre-trained large model.
[0021] Address extraction: Input the public opinion text data of the address to be extracted into the optimized pre-trained large model to automatically extract the address information in the text.
[0022] Preferably, in step S3, structurally decomposing the address information includes the following steps:
[0023] Address decomposition: decompose the extracted address information into base address and location description;
[0024] Direction analysis: Extracting direction information from direction descriptions.
[0025] Preferably, in step S4, the coordinate calculation includes the following steps:
[0026] Use the base address as the query keyword to query the map service interface to obtain the initial coordinates;
[0027] If there is no direction description in the address information, the initial coordinates are the target address coordinates.
[0028] If there is a direction description in the address information, the direction offset calculation is performed and the target address coordinates are obtained based on the direction offset calculation.
[0029] Preferably, the azimuth offset calculation includes the following steps:
[0030] Determine the offset parameters: convert the orientation information into offset distance and azimuth, and convert the initial coordinates into radians;
[0031] Calculate target coordinates: Calculate target longitude and latitude based on offset distance, azimuth and arc;
[0032] Get the target address coordinates: Convert the target latitude and longitude from radians to degrees to get the target address coordinates.
[0033] Preferably, the initial coordinates are converted into radians by the following formula:
[0034]
[0035]
[0036] in, is the latitude of the initial coordinate; is the longitude of the initial coordinate; Function to convert degrees to radians.
[0037] Preferably, the target latitude and longitude are calculated by the following formula:
[0038]
[0039]
[0040] in, is the target latitude; is the target longitude; is the offset distance; is the average radius of the Earth; is the azimuth.
[0041] Preferably, the conversion of the target latitude and longitude into an angle is calculated using the following formula:
[0042]
[0043]
[0044] in, Function to convert radians to degrees.
[0045] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0046] 1. This invention combines a pre-trained large model with supervised fine-tuning and few-sample prompts for text address extraction. By introducing a small number of examples and supervised fine-tuning, the model's recognition ability in the face of noise, colloquialisms, and ambiguous descriptions is enhanced, ensuring that the core address information in the text can be accurately extracted. The accuracy of address extraction in colloquial, noisy, and ambiguous texts is improved, significantly improving the accuracy and robustness of address recognition, and overcoming the obvious limitations of traditional rule-based or NER methods.
[0047] 2. The present invention obtains the base address and orientation description through address disassembly and orientation analysis, and then obtains the target address coordinates based on the base address and orientation description to achieve precise coordinate positioning of the target address, effectively solving the shortcomings of traditional technologies in processing additional orientation information and achieving fine positioning. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is a schematic flow diagram of the present invention. DETAILED DESCRIPTION
[0049] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described below with reference to the accompanying drawings and by way of preferred embodiments. However, it should be noted that many of the details listed in this specification are merely provided to help the reader gain a thorough understanding of one or more aspects of the present invention, and these aspects of the present invention can be practiced even without these specific details.
[0050] See also Figure 1 The present invention provides an address extraction method for public opinion text, and the technical solution is as follows:
[0051] A method for extracting addresses from public opinion texts, comprising the following steps:
[0052] S1. Data acquisition: Capture the public opinion text data of the address to be extracted from the public opinion platform.
[0053] Specifically, the data primarily comes from public opinion platforms, such as "Hanshan Wenzhong," which aggregate a large amount of textual information on user complaints and feedback. We regularly crawl textual data from these platforms using APIs or crawler technology to ensure coverage of all types of user feedback. We also annotate the data with information such as time, source, and user ID to facilitate subsequent tracing and statistical analysis.
[0054] S2. Large model address extraction: Input the public opinion text data into the pre-trained large model that has been optimized through supervised fine-tuning and few-sample prompts, and use the pre-trained large model to extract the address information of the public opinion text data.
[0055] Specifically, we use a large model to extract address information and optimize it for four common address expressions through supervised fine-tuning (SFT) and few-shot methods. The specific process is as follows:
[0056] Model supervised fine-tuning: Based on the pre-trained large model, supervised fine-tuning is performed using high-quality annotated data for address extraction tasks, making the model more adaptable to colloquial, noisy, and vaguely described text scenarios. During the supervised fine-tuning process, the model will learn names such as "Xiangti Lanwan" as the correct community names, and can recognize core address information such as "Xiangti Lanwan, Louqin North District". In addition, the fine-tuned model can accurately capture target information even when faced with public opinion texts containing irrelevant noise. Supervised fine-tuning is a method that uses a small amount of high-quality annotated data for targeted training based on a pre-trained large model, so that the model can better adapt to specific tasks. This method significantly improves the performance of the model in the target field by adjusting the model parameters, and has stronger semantic understanding and error correction capabilities.
[0057] Few-shot prompts: A small set of high-quality example prompts are designed for different expressions such as community names, intersections, unit names, and no addresses to guide the model to output standardized address information during the inference phase. In these examples, each prompt contains the input text and its corresponding standard output format, ensuring that the model can clearly understand the task requirements. For example, the model will learn to uniformly convert expressions such as "the intersection of XX Street and XX Road" into a standard format to correctly identify the address. After this few-shot prompt, the model can quickly capture key information and output a standardized address. Few-shot prompts are a technology that guides the model to understand task requirements and output formats by providing a small number of typical examples during the model inference phase. This method does not require a large amount of labeled data, and can enable the model to quickly master the required task patterns based on examples, thereby maintaining good generalization performance even when data is scarce.
[0058] Address extraction: The public opinion text of the address to be extracted is input into a large model that has been optimized through supervised fine-tuning and small-sample prompt design to automatically extract the core address information in the text.
[0059] For example, when faced with the text "The red light is long, causing traffic congestion. Xinggang Street (from south to north) motor vehicles turn left on the ground to Dushuhu Avenue...", the model optimized through supervised fine-tuning already has strong contextual understanding capabilities and can capture the address information "Xinggang Street (from south to north) motor vehicles turn left on the ground to Dushuhu Avenue" from the noisy text. The few-shot prompt provides the model with clear output format guidance during inference, prompting the model to present the extracted address in a standardized format, and ultimately output it as "the intersection of Xinggang Street and Dushuhu Avenue."
[0060] S3. Address decomposition and orientation analysis: Perform structured decomposition on the address information extracted from the large model to obtain the base address and orientation description.
[0061] Specifically, the address information extracted from the large model is structured and decomposed, which mainly includes the following steps:
[0062] Address decomposition: Extracted address text is decomposed into a base address and additional direction / distance descriptions. For example, "200 meters north of Xiangti Lanwan, North District, Loufeng" is decomposed into the base address "Xiangti Lanwan, North District, Loufeng" and the direction description "200 meters north".
[0063] Direction analysis: Using dictionary matching technology, we extract direction information (for example, "north" is extracted as "north") and distance values (for example, "200 meters" is extracted as "200") from the text to prepare for subsequent positioning.
[0064] S4. Address coordinate calculation: perform coordinate calculation based on the reference address and orientation description to obtain the target address coordinates.
[0065] Specifically, the decomposed base address is combined with the orientation description to obtain the latitude and longitude of the final target address. The main steps are as follows:
[0066] Base address positioning: Use the map service interface to use the base address as the query keyword to obtain its corresponding initial latitude and longitude , the initial latitude and longitude are used as the initial coordinates. Among them, Baidu Map API can be used as the map service interface.
[0067] If there is no azimuth offset description in the text, the initial coordinates are used as the target address coordinates.
[0068] If there is an azimuth offset description in the text, the azimuth offset calculation is performed based on the initial coordinates.
[0069] For example, by querying "Xiangti Lanwan, Loufeng North District", the system obtains the base coordinates:
[0070]
[0071] Azimuth offset calculation: For addresses containing azimuth descriptions, the base coordinates need to be corrected based on the extracted azimuth and offset distance. The specific process is as follows:
[0072] Determine the offset parameters: Convert the obtained orientation information into offset distance and azimuth , when the direction is due north, At the same time, convert the base coordinates to radians for the following calculations:
[0073]
[0074]
[0075] in, is the latitude of the initial coordinate; is the longitude of the initial coordinate; Function to convert degrees to radians.
[0076] Calculate target coordinates: Use the following formula to calculate the latitude and longitude of the target point, where the average radius of the earth is 6371000 meters.
[0077] Target latitude:
[0078]
[0079] Target longitude:
[0080]
[0081] in, is the target latitude; is the target longitude; is the offset distance; is the average radius of the Earth; is the azimuth, the average radius of the Earth Take 6371000 meters.
[0082] 3) Convert the target latitude and longitude from radians to degrees to get the final latitude and longitude:
[0083]
[0084]
[0085] in, Function to convert radians to degrees.
[0086] For example, "200 meters north of Xiangti Lanwan, Loufeng North District", the final coordinates are:
[0087]
[0088] Address offset calculation is based on a geographic coordinate offset calculation method. The latitude and longitude of the base address are first obtained through a map service. The base coordinates are then offset based on a predetermined direction (such as north, south, east, or west) and a specific distance to obtain the target coordinates. This offset calculation uses a spherical coordinate system to convert direction and distance into changes in longitude and latitude, achieving precise coordinate adjustments.
[0089] S5. Verification: Compare the calculated coordinates of the final target address with the true coordinates, and calculate the offset error and average error.
[0090] Specifically, first, the fine-tuned large model is used to extract address information from the text, including the basic address and direction description; then, the basic address is queried by calling the map service interface to obtain the initial longitude and latitude, and the vector offset is calculated in combination with the direction description to obtain the final positioning coordinates; finally, the coordinates calculated by the model are compared with the real coordinates, and the offset error and average error of each sample are counted to ensure positioning accuracy.
[0091] Based on a pre-trained large model, this method uses supervised fine-tuning and few-sample prompts to accurately extract noisy, colloquial, and ambiguous address information from public opinion texts. It fully leverages contextual semantics for automatic error correction and completion, significantly improving the robustness and accuracy of address extraction. It also performs coordinate correction based on reference coordinates combined with azimuth and distance information, calculating the precise coordinates of the target address and achieving precise location of the target address.
[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A method for extracting addresses from public opinion texts, characterized in that: The following steps are involved: S1. Data acquisition: crawling the public opinion text data of the address to be extracted from the public opinion platform; S2. Large model address extraction: Input the public opinion text data into the pre-trained large model that has been optimized through supervised fine-tuning and few-shot prompts, and use the pre-trained large model to extract the address information of the public opinion text data; S3. Address decomposition and orientation analysis: Perform structural decomposition on the address information extracted from the large model to obtain the base address and orientation description; S4. Address coordinate calculation: perform coordinate calculation based on the reference address and orientation description to obtain the target address coordinates; S5. Verification: Compare the calculated coordinates of the final target address with the true coordinates, and calculate the offset error and average error.
2. The method for extracting addresses from public opinion text according to claim 1, characterized in that: In step S1, the public opinion text data is text information of user complaints and feedback.
3. The method for extracting addresses from public opinion text according to claim 1, characterized in that: In step S2, extracting address information from public opinion text data includes the following steps: Model supervised fine-tuning: Based on the pre-trained large model, supervised fine-tuning is performed using high-quality labeled data for address extraction tasks; Few-sample prompts: Design a small set of high-quality example prompts for community names, intersections, unit names, and expressions without addresses to guide the pre-trained large model to output standardized address information during the inference phase, thereby obtaining an optimized pre-trained large model. Address extraction: Input the public opinion text data of the address to be extracted into the optimized pre-trained large model to automatically extract the address information in the text.
4. The method for extracting addresses from public opinion text according to claim 1, characterized in that: In step S3, the address information is structurally decomposed, including the following steps: Address decomposition: decompose the extracted address information into base address and location description; Direction analysis: Extracting direction information from direction descriptions.
5. The method for extracting addresses from public opinion text according to claim 4, characterized in that: In step S4, coordinate calculation includes the following steps: Use the base address as the query keyword to query the map service interface to obtain the initial coordinates; If there is no direction description in the address information, the initial coordinates are the target address coordinates; If there is a direction description in the address information, the direction offset calculation is performed and the target address coordinates are obtained based on the direction offset calculation.
6. The method for extracting addresses from public opinion text according to claim 5, characterized in that: The azimuth offset calculation comprises the following steps: Determine the offset parameters: convert the orientation information into offset distance and azimuth, and convert the initial coordinates into radians; Calculate target coordinates: Calculate target longitude and latitude based on offset distance, azimuth and arc; Get the target address coordinates: Convert the target latitude and longitude from radians to degrees to get the target address coordinates.
7. The method for extracting addresses from public opinion text according to claim 6, characterized in that: The conversion of the initial coordinates to radians is calculated by the following formula: in, is the latitude of the initial coordinate; is the longitude of the initial coordinate; Function to convert degrees to radians.
8. The method for extracting addresses from public opinion text according to claim 7, characterized in that: The target latitude and longitude are calculated by the following formula: in, is the target latitude; is the target longitude; is the offset distance; is the average radius of the Earth; is the azimuth.
9. The method for extracting addresses from public opinion text according to claim 8, characterized in that: The conversion of the target latitude and longitude into an angle is calculated using the following formula: in, Function to convert radians to degrees.