Text analysis method and device, equipment and storage medium
Through structured processing and point-of-interest recall technology, target points of interest are determined, and the problem of inaccurate text analysis in the existing technology is solved, the accuracy and consistency of address resolution are improved, and user operations are simplified.
Patent Information
- Application Number
- CN202411449697.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-01-03
AI Technical Summary
The prior art is not accurate enough in text analysis, especially when the address filled in the address bar may be inaccurate or blurred when the user places an order, it is impossible to effectively utilize the accurate address information in the remark area.
By obtaining the first point of interest and the text to be parsed, the structured process is performed to obtain the normalized field, the points of interest are recalled, the points of interest are filtered and integrated, and the target points of interest are determined based on the similarity to characterize the target address pointed to by the text to be parsed.
It improves the accuracy of text parsing, ensures the accuracy and consistency of address information, and simplifies user operation process.
Smart Images
Figure CN119250059A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a text parsing method, apparatus, device and storage medium. Background Art
[0002] With the development of mobile Internet and information technology, the way of freight transportation has changed a lot. At present, users can place orders through applications, and freight drivers accept orders on the applications and finally complete the loading, transportation and unloading of goods.
[0003] The ordering interface of the application includes an address bar and a note area. In one application scenario, the address filled in by the user in the address bar may be inaccurate or vague, and the text that can reflect the accurate address location is filled in the note area. For the above application scenario, the above text can be parsed to obtain the address of the user's order, and then the address can be automatically filled in the address bar to replace the original address, thereby simplifying the user operation.
[0004] However, in the prior art, keywords in the text are extracted only by relying on simple word segmentation technology, and then the keywords are filled into the address bar, which has the problem of inaccurate analysis of the text. Summary of the invention
[0005] The main purpose of this application is to provide a text parsing method, apparatus, computer device and storage medium, aiming to solve the technical problem of inaccurate text parsing.
[0006] To achieve the above purpose, the present application provides a text parsing method, which includes:
[0007] Acquire a first point of interest and a text to be parsed; the first point of interest is used to represent a preset first address;
[0008] Performing structural processing on the text to be parsed to obtain standardized fields;
[0009] Recalling points of interest on the normalized field to obtain N second points of interest, where N is a positive integer greater than 1;
[0010] Performing interest point filtering on the N second interest points to obtain M second interest points; the M second interest points are part of the N second interest points;
[0011] A target point of interest is determined according to the similarity between the first point of interest and each of the M second points of interest; the target point of interest is used to represent a target address pointed to by the text to be parsed.
[0012] Optionally, obtain the text to be parsed, including:
[0013] Get the text of the note area;
[0014] Detecting the character types included in the text;
[0015] If the character type included in the text meets the preset requirements, the text is determined to be a text to be parsed;
[0016] The preset requirements include at least one of the following:
[0017] The text includes Chinese;
[0018] The text does not include garbled characters;
[0019] The text does not include a phone number;
[0020] The text does not include special characters.
[0021] Optionally, the structural processing of the text to be parsed to obtain a normalized field includes:
[0022] Performing word segmentation on the text to be parsed to obtain a plurality of word segments included in the text to be parsed;
[0023] Removing noise words from the multiple word segments;
[0024] Structural annotation is performed on the multiple de-dried word segments to obtain the standardized fields.
[0025] Optionally, the recalling of interest points on the normalized field to obtain N second interest points includes:
[0026] Extracting city information included in the text to be parsed;
[0027] Querying a target keyword in a preset database; the target keyword includes the normalized field and the city information, and the database stores a mapping relationship between the keyword and the point of interest;
[0028] If a plurality of interest points corresponding to the target keyword are found in the database, the plurality of interest points are determined as the N second interest points.
[0029] Optionally, after searching the preset database for the target keyword, the method further includes:
[0030] If multiple points of interest corresponding to the target keyword are not found in the database, a self-built search and recall are performed on the target keyword to obtain multiple third points of interest, a geographic area search is performed on the target keyword to obtain multiple fourth points of interest, and a gate address type search is performed on the target keyword to obtain multiple fifth points of interest;
[0031] Deduplicating the plurality of third points of interest, the plurality of fourth points of interest, and the plurality of fifth points of interest;
[0032] The plurality of third interest points, the plurality of fourth interest points and the plurality of fifth interest points after deduplication are merged to obtain the N second interest points.
[0033] Optionally, filtering the N second points of interest to obtain M second points of interest includes:
[0034] calculating the longest common substring between the normalized field and each second point of interest name;
[0035] The M second interest points are obtained by removing some of the N second interest points whose corresponding longest common substring length is less than a preset value.
[0036] Optionally, determining the target interest point according to the similarity between the first interest point and each of the M second interest points includes:
[0037] Performing vectorization calculation on the first interest point to obtain a first vector value corresponding to the first interest point;
[0038] Performing vectorization calculation on each second interest point to obtain a second vector value corresponding to each second interest point;
[0039] Calculating a similarity value between the first vector value and each second vector value;
[0040] If the highest similarity value is greater than a preset threshold, the second interest point corresponding to the highest similarity value is determined as the target interest point.
[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a text parsing device, the text parsing device comprising:
[0042] An acquisition module, used to acquire a first point of interest and a text to be parsed; the first point of interest is used to represent a preset first address;
[0043] A processing module, used for performing structural processing on the text to be parsed to obtain a standardized field;
[0044] A recall module, used to recall points of interest on the normalized field to obtain N second points of interest; N is a positive integer greater than 1;
[0045] A filtering module, configured to filter the N second points of interest to obtain M second points of interest; the M second points of interest are part of the N second points of interest;
[0046] A determination module is used to determine a target interest point according to the similarity between the first interest point and each of the M second interest points; the target interest point is used to represent the target address pointed to by the text to be parsed.
[0047] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:
[0048] The computer device includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the text parsing methods proposed in the embodiments of the present application are implemented.
[0049] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:
[0050] The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of any one of the text parsing methods proposed in the embodiments of the present application are implemented.
[0051] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0052] The present application provides a text parsing method, apparatus, device and storage medium, the method comprising: obtaining a first point of interest and a text to be parsed; the first point of interest is used to represent a preset first address; the text to be parsed is structured to obtain a normalized field; the normalized field is recalled for points of interest to obtain N second points of interest; N is a positive integer greater than 1; the N second points of interest are filtered for points of interest to obtain M second points of interest; the M second points of interest are part of the N second points of interest; the target point of interest is determined according to the similarity between the first point of interest and each of the M second points of interest; the target point of interest is used to represent the target address pointed to by the text to be parsed. In the embodiment of the present application, the target point of interest representing the target address pointed to by the text to be parsed is determined by the above method, so as to improve the accuracy of text parsing. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0054] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;
[0055] Figure 2 is a flowchart of a text parsing method provided in an embodiment of the present application;
[0056] Figure 3 is an application flow chart of the text parsing method provided in an embodiment of the present application;
[0057] Figure 4 It is a structural diagram of an embodiment of a text parsing device provided in an embodiment of the present application;
[0058] Figure 5 This is a basic structural block diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION
[0059] The text parsing method provided in the embodiment of the present application is applied to a text parsing device. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" in the specification and claims of the present application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned figures are used to distinguish different objects, not to describe a specific order.
[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0061] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0062] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0063] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social online platform software, etc.
[0064] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.
[0065] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .
[0066] It should be noted that the text parsing method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the text parsing device is generally arranged in the server / terminal device.
[0067] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0068] Please refer to Figure 2 , shows a flow chart of an embodiment of the text parsing method proposed in the present application. The embodiment of the present application can acquire and process relevant data based on artificial intelligence technology.
[0069] The text parsing method provided in the embodiment of the present application comprises the following steps:
[0070] S210, obtaining a first point of interest and a text to be parsed.
[0071] A Point of Interest (POI) is a specific location on a map, including geographic coordinates and additional information.
[0072] In an optional application scenario, the order interface includes an address bar and a notes area. The user can enter an address in the address bar and enter notes in the notes area. The address filled in by the user in the address bar can be understood as the first point of interest, and the notes entered by the user in the notes box can be understood as text to be parsed.
[0073] S220, performing structural processing on the text to be parsed to obtain standardized fields.
[0074] In this step, the text to be parsed is subjected to structural processing, so that the text to be parsed is decomposed into fields. Since the above fields are obtained through structural processing, the above fields are standardized fields, that is, the fields meet the preset field requirements.
[0075] S230, recalling points of interest on the normalized field to obtain N second points of interest.
[0076] In this step, after the normalized field is obtained, interest points are recalled on the normalized field to obtain N second interest points.
[0077] The above-mentioned interest point recall operation is used to determine the interest points represented by the normalized field and determine the interest points associated with the normalized field.
[0078] Wherein, N is a positive integer greater than 1.
[0079] S240: Filter the N second points of interest to obtain M second points of interest.
[0080] In this step, in order to filter out irrelevant second interest points and improve the speed of subsequent interest point matching, the N second interest points are filtered to obtain M second interest points. For specific implementation methods, please refer to the subsequent embodiments.
[0081] The M second points of interest are some of the N second points of interest.
[0082] S250: Determine a target interest point according to a similarity between the first interest point and each of the M second interest points.
[0083] In this step, after obtaining the M second interest points, the similarity between the first interest point and each of the M second interest points is calculated, and the target interest point is determined according to the similarity between the first interest point and each of the second interest points.
[0084] The target point of interest is used to represent the target address pointed to by the text to be parsed.
[0085] In the embodiment of the present application, a first point of interest and a text to be parsed are obtained; the first point of interest is used to represent a preset first address; the text to be parsed is structured to obtain a normalized field; the normalized field is recalled for points of interest to obtain N second points of interest; N is a positive integer greater than 1; the N second points of interest are filtered to obtain M second points of interest; the M second points of interest are part of the N second points of interest; the target point of interest is determined based on the similarity between the first point of interest and each of the M second points of interest; the target point of interest is used to represent the target address pointed to by the text to be parsed. In the embodiment of the present application, the target point of interest representing the target address pointed to by the text to be parsed is determined in the above manner, and the address represented by the text is obtained, thereby improving the accuracy of text parsing.
[0086] Optionally, obtain the text to be parsed, including:
[0087] Get the text of the note area;
[0088] Detecting the character types included in the text;
[0089] If the character type included in the text meets the preset requirements, the text is determined to be a text to be parsed;
[0090] The preset requirements include at least one of the following:
[0091] The text includes Chinese;
[0092] The text does not include garbled characters;
[0093] The text does not include a phone number;
[0094] The text does not include special characters.
[0095] As described above, in an optional application scenario, the order interface includes a note area, and the user can enter notes in the note area. The notes entered by the user in the note box can be understood as text to be parsed.
[0096] In this embodiment, the text in the remarks area is obtained, and the character type included in the text is detected. If the character type included in the text meets the preset requirements, the text is determined to be the text to be parsed.
[0097] Optionally, the preset requirements include at least one of the following: the text includes Chinese; the text does not include garbled characters; the text does not include telephone numbers; the text does not include special characters.
[0098] Exemplarily, if the acquired text includes Chinese characters, does not include garbled characters, does not include telephone numbers, and does not include special characters, then the text is determined to be text to be parsed.
[0099] Exemplarily, if the acquired text does not include Chinese, it is determined that the text is not the text to be parsed.
[0100] Exemplarily, if the acquired text includes garbled characters, it is determined that the text is not the text to be parsed.
[0101] Exemplarily, if the acquired text includes a telephone number, it is determined that the text is not text to be parsed.
[0102] Exemplarily, if the acquired text includes special characters, it is determined that the text is not the text to be parsed.
[0103] In this embodiment, the text to be parsed is set as a text that meets preset requirements. By setting the format of the text to be parsed, it is avoided that in the subsequent text parsing process, the text parsing result is affected by too many interference factors in the text.
[0104] Optionally, the structural processing of the text to be parsed to obtain a normalized field includes:
[0105] Performing word segmentation on the text to be parsed to obtain a plurality of word segments included in the text to be parsed;
[0106] Removing noise words from the multiple word segments;
[0107] Structural annotation is performed on the multiple de-dried word segments to obtain the standardized fields.
[0108] In this embodiment, the text to be parsed may be first segmented to obtain multiple segmented words included in the text to be parsed. Further, noise reduction processing may be performed on the multiple segmented words to remove noise words from the multiple segmented words.
[0109] An optional implementation method is to apply an NLP algorithm to perform noise reduction on multiple word segments.
[0110] Another optional implementation is to pre-set a word library, which includes multiple noise words, query multiple word segments in the word library, obtain the noise words included in the multiple word segments, and remove the noise words.
[0111] Furthermore, the multiple de-noised segmented words are structurally annotated to obtain normalized fields. Optionally, the words representing addresses in the multiple segmented words may be annotated, for example, words representing information including provinces, cities, streets, communities, buildings, units, etc.
[0112] In this embodiment, by performing noise reduction processing on the segmented words and performing structured annotation on the multiple segmented words after noise reduction, it is avoided that in the subsequent text parsing process, the text parsing result is affected by too many interference factors in the text.
[0113] Optionally, the recalling of interest points on the normalized field to obtain N second interest points includes:
[0114] Extracting city information included in the text to be parsed;
[0115] Querying a target keyword in a preset database; the target keyword includes the normalized field and the city information, and the database stores a mapping relationship between the keyword and the point of interest;
[0116] If a plurality of interest points corresponding to the target keyword are found in the database, the plurality of interest points are determined as the N second interest points.
[0117] In this embodiment, a database is pre-set, and the database stores mapping relationships between keywords and points of interest.
[0118] The city information in the text to be parsed is extracted. Optionally, the word segmentation representing the city in the text to be parsed can be determined as the city information. The determined normalized field and the above city information are determined as target keywords.
[0119] The target keyword is queried in a preset database. If multiple points of interest corresponding to the target keyword are found in the database, it means that the points of interest associated with the text to be parsed have been cached, and the multiple points of interest are determined as N second points of interest.
[0120] Optionally, after searching the preset database for the target keyword, the method further includes:
[0121] If multiple points of interest corresponding to the target keyword are not found in the database, a self-built search and recall are performed on the target keyword to obtain multiple third points of interest, a geographic area search is performed on the target keyword to obtain multiple fourth points of interest, and a gate address type search is performed on the target keyword to obtain multiple fifth points of interest;
[0122] Deduplicating the plurality of third points of interest, the plurality of fourth points of interest, and the plurality of fifth points of interest;
[0123] The plurality of third interest points, the plurality of fourth interest points and the plurality of fifth interest points after deduplication are merged to obtain the N second interest points.
[0124] In this embodiment, if multiple points of interest corresponding to the target keyword are not found in the database, indicating that the points of interest associated with the text to be parsed are not pre-cached, then a self-built search recall can be performed on the target keyword to obtain multiple third points of interest, the target keyword can be recalled by geographic area to obtain multiple fourth points of interest, and the target keyword can be recalled by address type to obtain multiple fifth points of interest.
[0125] In order to consider factors such as performance, the recall results will be truncated in quantity, and the geographical area will be limited to ensure that the distance deviation of the analyzed points is small. Multiple third points of interest, multiple fourth points of interest, and multiple fifth points of interest are deduplicated and merged to obtain N second points of interest.
[0126] Specifically, the above-mentioned methods of self-built search and recall of target keywords include:
[0127] A vector feature library is pre-set, and the vector feature library stores the mapping relationship between vectors and interest points. Through a semantic indexing model, such as SimCSE and DiffCSE, the target keyword is vectorized to obtain a keyword vector corresponding to the target keyword, and a vector with the largest similarity value with the keyword vector among multiple vectors included in the vector feature library is determined, and the interest point with a mapping relationship with the vector is determined as the third interest point.
[0128] Specifically, the above-mentioned methods of geographic area recall of target keywords include:
[0129] The target keyword is semantically parsed to obtain a geographical area corresponding to the target keyword, and then the points of interest included in the geographical area are determined as fourth points of interest.
[0130] Specifically, the above-mentioned methods of recalling the target keyword by address type include:
[0131] A plurality of portal types and points of interest having a mapping relationship with each portal type are pre-set, and semantic analysis is performed on the target keyword to obtain the portal type corresponding to the target keyword, and then the point of interest having a mapping relationship with the portal type is determined as the fifth point of interest.
[0132] In this embodiment, N second points of interest are obtained by performing self-built search recall, geographic area recall and gate address type recall on the target keyword, so that the points obtained through the point of interest recall are more complete and accurate.
[0133] Optionally, filtering the N second points of interest to obtain M second points of interest includes:
[0134] calculating the longest common substring between the normalized field and each second point of interest name;
[0135] The M second interest points are obtained by removing some of the N second interest points whose corresponding longest common substring length is less than a preset value.
[0136] It should be understood that the longest common substring is the longest common substring of two or more known strings, and the longest common substring is continuous. For example, the longest common substring of the strings "ABABC", "BABCA" and "ABCBA" is "ABC".
[0137] In this embodiment, the longest common substring between the normalized field and each second point of interest name is calculated, and the length of the longest common substring is obtained. If the length of the longest common substring corresponding to the second point of interest is less than a preset value, the second point of interest is removed.
[0138] Optionally, in the scenario of application of this embodiment, the longest common substring length of 2 is generally building information, such as Building A, Building 1, etc., but the building information is the same. In this case, the second interest points corresponding to the longest common substring length less than 2 are deleted to obtain M second interest points.
[0139] In this embodiment, according to the longest common substring corresponding to the second point of interest, the range of the target point of interest can be greatly narrowed, the workload of subsequent similarity calculation can be reduced, the speed of similarity calculation can be increased, the system operating resources can be reduced, and the efficiency of acquiring the target point of interest can be improved.
[0140] Optionally, determining the target interest point according to the similarity between the first interest point and each of the M second interest points includes:
[0141] Performing vectorization calculation on the first interest point to obtain a first vector value corresponding to the first interest point;
[0142] Performing vectorization calculation on each second interest point to obtain a second vector value corresponding to each second interest point;
[0143] Calculating a similarity value between the first vector value and each second vector value;
[0144] If the highest similarity value is greater than a preset threshold, the second interest point corresponding to the highest similarity value is determined as the target interest point.
[0145] In this embodiment, an optional implementation method is to perform TF IDF vectorization calculation on the first interest point to obtain three dimensional values; concatenate the three dimensional values to obtain a first vector value corresponding to the first interest point; perform TF IDF vectorization calculation on each second interest point to obtain three dimensional values; concatenate the three dimensional values to obtain a second vector value corresponding to each second interest point.
[0146] Among them, TF IDF (term frequency-inverse document frequency) is a commonly used weighting technique for information retrieval and data mining. TF stands for term frequency, and IDF stands for inverse document frequency. TF IDF is a statistical method used to evaluate the importance of a word to a document set or one of the documents in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a document, but at the same time it decreases in inverse proportion to the frequency of its appearance in the corpus. Various forms of TF IDF weighting are often used by search engines as a measure or rating of the relevance between a document and a user query.
[0147] It should be understood that the interest points may also be vectorized and calculated in other ways to obtain vector values, which are not specifically limited here.
[0148] Furthermore, the physical distance between the first vector value and each second vector value is calculated, and the cosine similarity between the retrieved vector values is calculated according to the cosine formula, and the cosine similarity is used as the similarity value between the first vector value and each second vector value.
[0149] In this embodiment, a threshold is pre-set, and the maximum similarity value corresponding to the M second vector values is determined. The maximum similarity value is compared with the preset threshold. If the highest similarity value is greater than the preset threshold, it means that the point of interest corresponding to the maximum similarity value can represent an accurate address, and the point of interest is determined as the target point of interest.
[0150] Optionally, a lower threshold is also set. If the maximum similarity value is less than the lower threshold, it means that the point of interest corresponding to the maximum similarity value cannot represent an accurate address.
[0151] In an embodiment of the present application, the target point of interest is determined by calculating the similarity value between the first vector value and each second vector value, and comparing the size relationship between the highest similarity value and the preset threshold. The similarity between the address represented by each second point of interest in the embodiment of the present application and the address pre-filled by the user is used to determine an accurate and reliable target point of interest.
[0152] To understand the overall technical solution, please refer to Figure 3 ,like Figure 3As shown, the text to be parsed is obtained; the text to be parsed is structured to obtain normalized fields; the normalized fields and the city information included in the text to be parsed are used as target keywords; the database is queried to see whether there are multiple points of interest corresponding to the target keywords; if so, the multiple points of interest are determined as N second points of interest; if not, the target keywords are self-built search recall, geographic area recall and gate address type recall are performed to obtain N second points of interest. The N second points of interest are filtered to obtain M second points of interest; the M second points of interest are matched to determine the target point of interest.
[0153] Among them, the above-mentioned interest point matching process is: calculate the similarity between the pre-acquired first interest point and each second interest point, perform vectorization calculation on the first interest point, and obtain the first vector value corresponding to the first interest point; perform vectorization calculation on each second interest point to obtain the second vector value corresponding to each second interest point; calculate the similarity value between the first vector value and each second vector value; if the highest similarity value is greater than the preset threshold, the second interest point corresponding to the highest similarity value is determined as the target interest point, so as to achieve matching between the first interest point and each second interest point.
[0154] See also Figure 4 , an embodiment of the present application provides a text parsing device 400, the text parsing device 400 comprising:
[0155] The acquisition module 410 is used to acquire a first point of interest and a text to be parsed; the first point of interest is used to represent a preset first address;
[0156] The processing module 420 is used to perform structural processing on the text to be parsed to obtain a normalized field;
[0157] The recall module 430 is used to recall the interest points of the normalized field to obtain N second interest points; N is a positive integer greater than 1;
[0158] The filtering module 440 is used to filter the N second points of interest to obtain M second points of interest; the M second points of interest are part of the N second points of interest;
[0159] The determination module 450 is used to determine a target interest point according to the similarity between the first interest point and each of the M second interest points; the target interest point is used to represent the target address pointed to by the text to be parsed.
[0160] Optionally, the acquisition module 410 is specifically configured to:
[0161] Get the text of the note area;
[0162] Detecting the character types included in the text;
[0163] If the character type included in the text meets the preset requirements, the text is determined to be a text to be parsed;
[0164] The preset requirements include at least one of the following:
[0165] The text includes Chinese;
[0166] The text does not include garbled characters;
[0167] The text does not include a phone number;
[0168] The text does not include special characters.
[0169] Optionally, the processing module 420 is specifically configured to:
[0170] Performing word segmentation on the text to be parsed to obtain a plurality of word segments included in the text to be parsed;
[0171] Removing noise words from the multiple word segments;
[0172] Structural annotation is performed on the multiple de-dried word segments to obtain the standardized fields.
[0173] Optionally, the recall module 430 is specifically configured to:
[0174] Extracting city information included in the text to be parsed;
[0175] Querying a target keyword in a preset database; the target keyword includes the normalized field and the city information, and the database stores a mapping relationship between the keyword and the point of interest;
[0176] If a plurality of interest points corresponding to the target keyword are found in the database, the plurality of interest points are determined as the N second interest points.
[0177] Optionally, the recall module 430 is further specifically configured to:
[0178] If multiple points of interest corresponding to the target keyword are not found in the database, a self-built search and recall are performed on the target keyword to obtain multiple third points of interest, a geographic area search is performed on the target keyword to obtain multiple fourth points of interest, and a gate address type search is performed on the target keyword to obtain multiple fifth points of interest;
[0179] Deduplicating the plurality of third points of interest, the plurality of fourth points of interest, and the plurality of fifth points of interest;
[0180] The plurality of third interest points, the plurality of fourth interest points and the plurality of fifth interest points after deduplication are merged to obtain the N second interest points.
[0181] Optionally, the filtering module 440 is specifically configured to:
[0182] calculating the longest common substring between the normalized field and each second point of interest name;
[0183] The M second interest points are obtained by removing some of the N second interest points whose corresponding longest common substring length is less than a preset value.
[0184] Optionally, the determining module 450 is specifically configured to:
[0185] Performing vectorization calculation on the first interest point to obtain a first vector value corresponding to the first interest point;
[0186] Performing vectorization calculation on each second interest point to obtain a second vector value corresponding to each second interest point;
[0187] Calculating a similarity value between the first vector value and each second vector value;
[0188] If the highest similarity value is greater than a preset threshold, the second interest point corresponding to the highest similarity value is determined as the target interest point.
[0189] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.
[0190] The computer device 5 includes a memory 51, a processor 52, and a network interface 53 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 5 with components 51-53, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.
[0191] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0192] The memory 51 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 51 can be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, the memory 51 can also be an external storage device of the computer device 5, such as a plug-in hard disk equipped on the computer device 5, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 51 can also include both the internal storage unit of the computer device 5 and its external storage device. In this embodiment, the memory 51 is generally used to store the operating system and various application software installed on the computer device 5, such as the program code of the text parsing method, etc. In addition, the memory 51 can also be used to temporarily store various types of data that have been output or are to be output.
[0193] The processor 52 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 52 is generally used to control the overall operation of the computer device 5. In this embodiment, the processor 52 is used to run the program code stored in the memory 51 or process data, such as running the program code of the text parsing method.
[0194] The network interface 53 may include a wireless network interface or a wired network interface. The network interface 53 is generally used to establish a communication connection between the computer device 5 and other electronic devices.
[0195] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores the text parsing program, and the text parsing program can be executed by at least one processor so that the at least one processor performs the steps of the text parsing method as described above.
[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware online platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0197] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0198] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A text parsing method, characterized in that: include: Get the first point of interest and the text to be parsed; The first point of interest is used to represent a preset first address; Performing structural processing on the text to be parsed to obtain standardized fields; Recalling points of interest on the normalized field to obtain N second points of interest, where N is a positive integer greater than 1; Performing interest point filtering on the N second interest points to obtain M second interest points; the M second interest points are part of the N second interest points; A target point of interest is determined according to the similarity between the first point of interest and each of the M second points of interest; the target point of interest is used to represent a target address pointed to by the text to be parsed.
2. The method according to claim 1, characterized in that Get the text to be parsed, including: Get the text of the note area; Detecting the character types included in the text; If the character type included in the text meets the preset requirements, the text is determined to be a text to be parsed; The preset requirements include at least one of the following: The text includes Chinese; The text does not include garbled characters; The text does not include a phone number; The text does not include special characters.
3. The method according to claim 1, characterized in that The structural processing of the text to be parsed to obtain a standardized field includes: Performing word segmentation on the text to be parsed to obtain a plurality of word segments included in the text to be parsed; Removing noise words from the multiple word segments; Structural annotation is performed on the multiple de-dried word segments to obtain the standardized fields.
4. The method according to claim 1, characterized in that: The step of recalling points of interest on the normalized field to obtain N second points of interest includes: Extracting city information included in the text to be parsed; Querying a target keyword in a preset database; the target keyword includes the normalized field and the city information, and the database stores a mapping relationship between the keyword and the point of interest; If a plurality of interest points corresponding to the target keyword are found in the database, the plurality of interest points are determined as the N second interest points.
5. The method according to claim 4, characterized in that After searching the preset database for the target keyword, the method further includes: If multiple points of interest corresponding to the target keyword are not found in the database, a self-built search and recall are performed on the target keyword to obtain multiple third points of interest, a geographic area search is performed on the target keyword to obtain multiple fourth points of interest, and a gate address type search is performed on the target keyword to obtain multiple fifth points of interest; Deduplicating the plurality of third points of interest, the plurality of fourth points of interest, and the plurality of fifth points of interest; The plurality of third interest points, the plurality of fourth interest points and the plurality of fifth interest points after deduplication are merged to obtain the N second interest points.
6. The method according to claim 1, characterized in that The filtering the N second points of interest to obtain M second points of interest includes: calculating the longest common substring between the normalized field and each second point of interest name; The M second interest points are obtained by removing some of the N second interest points whose corresponding longest common substring length is less than a preset value.
7. The method according to any one of claims 1 to 6, characterized in that The determining the target interest point according to the similarity between the first interest point and each of the M second interest points includes: Performing vectorization calculation on the first interest point to obtain a first vector value corresponding to the first interest point; Performing vectorization calculation on each second interest point to obtain a second vector value corresponding to each second interest point; Calculating a similarity value between the first vector value and each second vector value; If the highest similarity value is greater than a preset threshold, the second interest point corresponding to the highest similarity value is determined as the target interest point.
8. A text analysis device, characterized in that: include: An acquisition module, used for acquiring the first point of interest and the text to be parsed; The first point of interest is used to represent a preset first address; A processing module, used for performing structural processing on the text to be parsed to obtain a standardized field; A recall module, used to recall points of interest on the normalized field to obtain N second points of interest; N is a positive integer greater than 1; A filtering module, configured to filter the N second points of interest to obtain M second points of interest; the M second points of interest are part of the N second points of interest; A determination module is used to determine a target interest point according to the similarity between the first interest point and each of the M second interest points; the target interest point is used to represent the target address pointed to by the text to be parsed.
9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the text parsing method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the text parsing method according to any one of claims 1 to 7 are implemented.