A general data fuzzy search and matching method based on FuzzyWuzzy algorithm

Abstract

The present application provides a general data fuzzy search and matching method based on the FuzzyWuzzy algorithm, which relates to the field of search and matching, including: receiving keywords and dimension parameters input by a user, analyzing the dimension parameters, and determining a priority; matching the keywords with a database according to the priority to generate a candidate result set, and reordering the candidate result set according to the dimension parameters to obtain a final result set; performing a dimension check on the final result set, returning data if it passes, and re-executing the matching process or reporting an error if it fails.

Description

Technical Field

[0001] The present application belongs to the technical field of search and matching, and specifically relates to a general data fuzzy search and matching method based on the FuzzyWuzzy algorithm. Background Art

[0002] With the rapid development of information technology, the digital transformation of enterprises has become an irreversible trend. As the core facility supporting various business applications, the functionality and ease of use of enterprise-level cloud platforms directly affect the operational efficiency of enterprises. As a comprehensive management tool serving enterprise users, GSCloud Enterprise Digital Cloud Platform provides a series of services covering data management, process control and report generation. In order to further optimize the user experience, the platform has specially integrated a powerful search function, allowing users to quickly locate the required information through fuzzy search. However, with the advancement of cloud computing technology and the enhancement of big data analysis capabilities, enterprises have also increased their requirements for search functions. Traditional precise matching methods are no longer sufficient to cope with the growing demand. Fuzzy search is particularly important because it can better understand and meet the actual intentions of users. It requires the system to handle not only standard queries, but also non-standard inputs such as spelling errors, synonyms and abbreviations.

[0003] Currently, the fuzzy search function on the GSCloud platform is mainly implemented through the SQL query language, especially using the LIKE operator to complete fuzzy matching tasks. However, when the query conditions contain wildcards, the LIKE operator often results in a full table scan rather than accelerating the query through indexes, which significantly increases the response time when processing large-scale data sets and complex queries. On the other hand, in some advanced application scenarios, the platform adopts an application-level direct data comparison method. Although this method can speed up the response in some scenarios, it is also accompanied by high memory consumption and maintenance costs. In addition, if the search weight or matching algorithm is not properly configured, the search results may not be accurate enough, especially when dealing with diverse user input forms, the quality of the search may decline. Summary of the invention

[0004] The present application provides a general data fuzzy search and matching method based on the FuzzyWuzzy algorithm to solve the problems of slow response speed, high memory consumption and inaccurate search in processing large-scale data queries.

[0005] The technical solution adopted in this application is:

[0006] The present application embodiment provides a general data fuzzy search and matching method based on the FuzzyWuzzy algorithm, comprising:

[0007] Receiving keywords and dimension parameters input by a user, analyzing the dimension parameters, and determining priorities;

[0008] According to the priority, the keywords are matched with the database to generate a candidate result set, and according to the dimension parameter, the candidate result set is reordered to obtain a final result set;

[0009] The final result set is subjected to a dimension check. If it passes, the data is returned. If it fails, the matching process is re-executed or an error is reported.

[0010] The general data fuzzy search and matching method based on the FuzzyWuzzy algorithm provided in the present application also includes the following additional technical features: receiving keywords and dimension parameters input by the user, analyzing the dimension parameters, and determining the priority, specifically:

[0011] Accept the search character set, dimension information and dimension order input by the user, determine the importance of each dimension and the role of each dimension in the matching process according to the dimension information, and sort the dimensions according to priority according to the dimension order.

[0012] According to an embodiment of the present application, the keyword is matched with the database according to the priority to generate a candidate result set, and the candidate result set is reordered according to the dimension parameter to obtain a final result set, specifically:

[0013] The search character set input by the user is split into multiple parts, and for each split part, the FuzzyWuzzy algorithm is used to match the target character set to generate a series of result sets; all result sets are merged and duplicates are removed to form a preliminary candidate result set; according to the dimension information previously determined, the candidate result sets are reordered to obtain a final result set.

[0014] According to one embodiment of the present application, the final result set is subjected to a dimensionality check, and if it passes, data is returned; if it fails, the matching process is re-executed or an error is reported, specifically: the correctness and relevance of the matching result are checked according to the dimensional parameters passed in; if the result is correct, data is prepared to be returned; if the result is incorrect, the similarity is not high enough or is irrelevant to the search content, the matching process needs to be re-executed or an error message is returned.

[0015] According to an embodiment of the present application, the returned data specifically includes: returning the sorted final result set to the user.

[0016] According to an embodiment of the present application, the returning error information is specifically: if any problem is found during the search process, a corresponding error message is returned to prompt the user of the problem or suggest re-entering the search conditions.

[0017] According to an embodiment of the present application, the FuzzyWuzzy algorithm is used to evaluate the similarity between two character strings, and the FuzzyWuzzy algorithm includes an edit distance algorithm and a Jaccard similarity.

[0018] A computer program product containing instructions, when running on a device, enables the device to execute the steps in a general data fuzzy search and matching method based on a FuzzyWuzzy algorithm.

[0019] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of a general data fuzzy search and matching method based on a FuzzyWuzzy algorithm.

[0020] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor. When the processor executes the program, steps in a general data fuzzy search and matching method based on a FuzzyWuzzy algorithm are implemented.

[0021] Due to the adoption of the above technical solution, the beneficial effects achieved by this application are as follows:

[0022] 1. By introducing the FuzzyWuzzy algorithm, the similarity between two strings can be evaluated based on a variety of similarity measurement methods (such as edit distance algorithm, Jaccard similarity, etc.), so as to more accurately identify the content that matches the user input in the massive data. Compared with the traditional precise matching method, this method can better understand and meet the user's actual intention, and even if the user input contains spelling errors or uses synonyms, abbreviations, etc., it can also obtain more accurate search results.

[0023] 2. This technical solution allows users to customize dimension parameters and dimension order, so that they can flexibly adjust the search strategy according to specific needs. This flexibility enables the system to adapt to diverse and complex input forms, providing users with a personalized search experience and enhancing user satisfaction.

[0024] 3. The FuzzyWuzzy algorithm can still maintain a fast matching speed when processing large amounts of data, effectively solving the full table scan problem caused by the traditional SQL LIKE operator. Through priority sorting, the system can filter out high-quality result sets in a short time, reducing unnecessary computing resource consumption, greatly shortening response time, and improving the overall efficiency of the system.

[0025] 4. By checking the dimensions of the final result set, the correctness and relevance of the returned data are ensured. If any problems are found, the system will automatically re-execute the matching process or report an error, avoiding misjudgment due to data quality issues. This mechanism improves the stability and reliability of the system.

[0026] 5. This method not only improves the search function, but also indirectly promotes the modernization of enterprise data management. By optimizing the search function, the system can process data more efficiently, helping enterprises to better utilize information resources, support decision-making, and improve operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0028] Figure 1 A flowchart of a general data fuzzy search and matching method based on the FuzzyWuzzy algorithm provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to more clearly illustrate the overall concept of the present application, a detailed description is given below in an illustrative manner in conjunction with the accompanying drawings.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below. It should be noted that the embodiments of the present application and the features in each embodiment may be combined with each other without conflict.

[0031] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or a communication; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0032] In the present application, unless otherwise clearly specified and limited, a first feature "above" or "below" a second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in an appropriate manner in any one or more embodiments or examples.

[0033] Example 1

[0034] like Figure 1 As shown, a general data fuzzy search and matching method based on the FuzzyWuzzy algorithm includes:

[0035] Step S100: receiving keywords and dimension parameters input by a user, analyzing the dimension parameters, and determining priorities.

[0036] Specifically, receive the keywords input by the user:

[0037] Keywords: This is the information that the user wants to obtain through the search, such as "financial report". The keywords entered by the user may be a single word or a short phrase, and the system needs to accurately capture these inputs as the basis for the search.

[0038] Example: The user enters "financial report" and the system needs to recognize and save this search string.

[0039] Receive dimension parameters:

[0040] Dimension parameters: These parameters provide additional guidance information for search, helping the system better understand and process user queries. Dimension parameters can include but are not limited to similarity thresholds, usage frequency, timestamps, etc.

[0041] For example, a user may specify a similarity threshold of 80%, which means that search results must be at least 80% similar; a usage frequency of high means that frequently mentioned content is displayed first; and a timestamp of the last week means that only updates within the last week are considered.

[0042] Analysis dimension parameters:

[0043] Dimension information: The system needs to parse the dimension parameters provided by the user and understand the specific meaning and function of each dimension. For example, the "similarity" dimension may involve the degree of string matching, and the "frequency of use" dimension may involve the number of times a specific content appears in the data set.

[0044] Example: If the user specifies a similarity threshold of 80%, the system needs to ensure that the similarity of the matching results is not less than this threshold.

[0045] Prioritize:

[0046] Dimension order: The user-entered dimension parameter may include a dimension order, indicating which dimensions should be prioritized during the matching process.

[0047] Example: A user might specify that similarity should be prioritized over frequency, which means that the system matches based on similarity first, then considers frequency if similarity is the same.

[0048] The specific operations are as follows:

[0049] Receive keywords:

[0050] The user enters a search keyword, such as "financial report", through the interface, and the system stores it as a search character set.

[0051] Receive dimension information:

[0052] The user simultaneously inputs or selects dimension information, such as a similarity threshold of 80%, high frequency of use, and a timestamp of the most recent week, and the system stores it as dimension information.

[0053] Receive dimension order:

[0054] The user specifies the dimension order, such as similarity over frequency of use, and the system stores it as the dimension order.

[0055] Analysis dimension parameters:

[0056] The system analyzes the passed dimension parameters, for example:

[0057] Similarity threshold: The system recognizes that users want the matching results to be at least 80% similar.

[0058] Frequency of use: The system recognizes that users want to prioritize frequently appearing content.

[0059] Timestamp: The system recognizes that the user wants to consider only updates within the last week.

[0060] Prioritize:

[0061] Based on the dimension order, the system sorts the dimensions by priority:

[0062] Similarity takes precedence over frequency of use.

[0063] In the case of equal similarity, the content with higher frequency of use takes precedence.

[0064] In the specific implementation scenario of the present invention, receiving the keyword input by the user

[0065] Suppose a user wants to search for information about "financial reporting" on the GSCloud enterprise digital cloud platform.

[0066] Keywords: The search term entered by the user is "financial report".

[0067] Receive dimension parameters

[0068] Users can select or enter multiple dimension parameters to further refine the search results. Dimension parameters include but are not limited to:

[0069] Similarity: Users hope that the similarity between the search results and the entered keywords reaches a certain threshold.

[0070] Frequency of use: Users may be concerned about how often certain keywords appear in a document.

[0071] Timestamp: Users may want search results to be the most recently updated documents.

[0072] For example, the user might enter the following dimension parameters:

[0073] Similarity: At least 80% similarity.

[0074] Frequency of use: Documents that have been cited the most times in the past year.

[0075] Timestamp: Updated within the last month.

[0076] Analysis Dimension Parameters

[0077] The system needs to analyze the dimension parameters entered by the user to determine the specific meaning of each dimension and its role in the matching process. For example:

[0078] Similarity: The system will use the FuzzyWuzzy algorithm to calculate the similarity between strings and ensure that the similarity of the search results is not less than 80%.

[0079] Frequency of use: The system will count the number of times each document has been cited in the past year and give priority to documents with more citations.

[0080] Timestamp: The system will filter out documents that were not updated within the last month.

[0081] Prioritize

[0082] The user may also specify an order of precedence for dimension parameters, for example:

[0083] Priority order: similarity > usage frequency > timestamp.

[0084] This means that the system first considers similarity during the matching process, and if multiple documents have the same similarity, it further considers the frequency of use and finally the timestamp.

[0085] Specific operations

[0086] Receive keywords:

[0087] The user enters the search term "financial report" through the interface.

[0088] Receive dimension information:

[0089] The user enters or selects dimension information through the interface, for example:

[0090] Similarity: At least 80% similarity.

[0091] Frequency of use: Documents that have been cited the most times in the past year.

[0092] Timestamp: Updated within the last month.

[0093] Receive dimension order:

[0094] The user specifies the dimension order through the interface, for example: similarity > usage frequency > timestamp.

[0095] Analysis dimension parameters:

[0096] The system analyzes the incoming dimension parameters:

[0097] Similarity: The system recognizes that users want matching results to be at least 80% similar.

[0098] Frequency of use: The system recognizes that the user wants to prioritize documents that have been cited more frequently in the past year.

[0099] Timestamp: The system recognizes that the user wishes to consider only updates within the last month.

[0100] Prioritize: Based on the dimension order, the system prioritizes the dimensions: similarity takes precedence over frequency. In the case of equal similarity, documents with higher frequency take precedence. Lastly, timestamps are considered.

[0101] Through this series of operations, the system can accurately receive the user's search request and determine the matching rules in the search process based on the user's specific needs and preferences, thereby providing search results that are more in line with the user's expectations.

[0102] It should be noted that, based on the above solution, the system can automatically recommend appropriate dimension parameter combinations according to historical search behaviors and user preferences, thereby reducing the workload of manual configuration by users.

[0103] Furthermore, more dimensional information can be introduced, such as geographic location, timestamp, user behavior trajectory, etc., to make search results more personalized and contextual.

[0104] Furthermore, natural language processing technology can be combined to perform semantic analysis on the search character set and extract keywords and phrases, so as to more accurately understand the user's search intent.

[0105] Furthermore, it can also provide instant feedback during the user input process, displaying the results with the highest similarity to the input content, and help users adjust their search strategies in a timely manner.

[0106] Furthermore, users may be allowed to rate or mark search results, and the system collects such feedback for subsequent search optimization.

[0107] Furthermore, the user's search history can be analyzed to recommend search dimensions and sequences that may be of interest to the user based on the user's behavior patterns.

[0108] Furthermore, based on the user's input habits, possible search intentions can be predicted, relevant dimensional information can be loaded in advance, and the search process can be accelerated.

[0109] Furthermore, users may be allowed to specify multiple data sources, and the system can extract information from data from different sources to enrich search results.

[0110] Furthermore, third-party service APIs, such as weather forecasts, news information, etc., can be integrated to add external information dimensions to search results.

[0111] Step S200: Match the keywords with the database according to the priority to generate a candidate result set, and reorder the candidate result set according to the dimension parameters to obtain a final result set.

[0112] Generate candidate result set according to priority matching

[0113] When the user enters keywords and defines related dimension parameters, the system will first determine the priority order based on these dimension parameters. Then, the system will match the keywords according to this priority order and generate a preliminary candidate result set. The specific process is as follows:

[0114] Receiving input: The system receives the search character set, dimension information (such as similarity, frequency of use, etc.) and dimension order (such as similarity takes precedence over frequency of use) input by the user.

[0115] Prioritize: Determine the importance of each dimension based on the dimension information and prioritize the dimensions according to the dimension order.

[0116] Generate candidate result set: The system splits the search character set entered by the user into multiple parts, and uses the FuzzyWuzzy algorithm to match each part with the target character set in the database. In this way, a result set is generated for each split part. Next, the system merges all these result sets together and removes duplicates to form a preliminary candidate result set.

[0117] Reorder the result set according to the dimension parameters

[0118] Once the candidate result set is obtained, the next step is to reorder these candidate results according to the dimension parameters to obtain the final result set. The process is as follows:

[0119] Dimension check: Check each record in the candidate result set according to the dimension parameters entered by the user. The check includes the correctness and relevance of the matching results.

[0120] Sorting and filtering: Reorder the candidate result set based on the previously determined dimension information. For example, if similarity is the most prioritized dimension, the system will first sort the result set based on similarity.

[0121] Generate the final result set: Finally, the system will reorder the candidate result sets according to the priority order of the dimension parameters and generate the final result set.

[0122] Suppose the user enters the keyword "financial report" and sets the dimension parameters to "similarity" and "frequency of use", and stipulates that similarity takes precedence over frequency of use. Then the system will first match the keyword "financial report" based on similarity and find all documents similar to it. Next, the system will sort these similar documents by frequency of use, that is, those documents that have been cited more in the past will be placed in a higher position. Finally, the system will generate a final result set that takes both similarity and frequency of use into account and return it to the user.

[0123] In the specific implementation scenario of the present invention, it is assumed that a user searches for information related to "financial reports" on the GSCloud enterprise digital cloud platform, and the user also sets two dimension parameters: similarity and frequency of use, where similarity takes precedence over frequency of use. The following are the specific steps:

[0124] Receive input: The user enters the search keyword "financial report" and sets the dimension parameters to similarity and frequency of use, with similarity taking precedence over frequency of use.

[0125] Determine priority: The system determines the priority order based on the dimension parameters set by the user, that is, similarity is considered first in the matching process, followed by frequency of use.

[0126] Generate candidate result set:

[0127] The system splits the user's search keyword "financial report" into individual terms: "finance" and "report".

[0128] For each word, the system uses the FuzzyWuzzy algorithm to match the target character set in the database. For example, for the word "finance", the system finds all document titles or content fragments in the database that are similar to "finance". It can be assumed that the FuzzyWuzzy algorithm uses the edit distance algorithm to calculate similarity.

[0129] Similarly, for the word "report", the system will find all document titles or content fragments that are similar to it.

[0130] The matching results of the two terms are merged and duplicates are removed to form a preliminary candidate result set.

[0131] Reorder to get the final result set:

[0132] Based on the dimension parameters determined previously, i.e., similarity takes precedence over frequency of use, the system first sorts each entry in the candidate result set according to similarity.

[0133] Entries with the same similarity are sorted according to their frequency of use.

[0134] Finally, the system returns the sorted result set to the user, which is the final result set.

[0135] Specific Examples

[0136] Assume that there are the following documents in the database:

[0137] Document A: Title “2020 Financial Report”, similarity score 85%, medium frequency of use.

[0138] Document B: Title “Financial Statement Analysis”, similarity score 70%, high frequency of use.

[0139] Document C: Title “2021 Annual Financial Summary”, similarity score 90%, average frequency of use.

[0140] Document D: Title “Financial Budget Preparation”, similarity score 75%, low frequency of use.

[0141] The order after sorting by similarity is: Document C>Document A>Document B>Document D.

[0142] Under the premise of the same similarity, the documents are sorted again according to the frequency of use. However, in this example, the similarity score of each document is unique, so the final sorting result is: Document C>Document A>Document B>Document D.

[0143] In the final result set, document C is ranked first because it has the highest similarity score, even though it is not the most frequently used document; document B is ranked third despite its high frequency of use, due to its low similarity. In this way, the system can return the document that best matches the user's search intent.

[0144] It should be noted that, based on the above solution, a cache mechanism can be used to save the matching results of frequent queries to reduce the computational burden of each search.

[0145] Furthermore, an inverted index or other efficient data structures may be used to speed up the search process.

[0146] Furthermore, in a distributed environment, search requests can be distributed to different nodes for parallel processing, thereby increasing the search speed.

[0147] Furthermore, it is possible to combine machine learning models to train a smarter prediction model to estimate the user's search intent and thus match the results more accurately.

[0148] Furthermore, natural language processing techniques can be used to enhance search capabilities, such as entity recognition and sentiment analysis, to improve the relevance of search results.

[0149] Furthermore, context awareness can be introduced to adjust the priority of search results based on factors such as the user's search history and geographic location.

[0150] Step S300: perform a dimension check on the final result set. If it passes, the data is returned. If it fails, the matching process is re-executed or an error is reported.

[0151] Dimension check: At this stage, the system will check the generated candidate result set based on the dimension parameters provided by the user. Dimension parameters refer to the standards or rules used to guide the search process, which may include but are not limited to the importance of keywords, search priority, etc. The purpose of the check is to verify whether the result set meets the pre-set dimension requirements, such as whether it meets the minimum similarity threshold, whether it is relevant to the search content, etc.

[0152] Correctness verification: The system evaluates the correctness of search results. If the result is correct, it means that the information searched is what the user expects, or is similar enough to the query entered by the user. If the result is incorrect, it may be due to an error in the search process, or because the result does not meet the user's expectations, such as the similarity is too low or completely irrelevant.

[0153] Relevance confirmation: In addition to correctness, the relevance of search results also needs to be verified. Even if the search results are technically correct, if they do not match the user's needs, then such results are worthless. Relevance checks help ensure that the information returned to users is what they really care about.

[0154] Return data: If the dimension check indicates that the search results are both correct and relevant, then the system returns those results to the user. This usually means returning a list of results sorted in some way (such as in descending order of similarity).

[0155] Re-execute the matching process or report an error: If the check finds that the results are incorrect or not relevant enough, the system has two options. One is to re-execute the entire search and matching process to try to find a better result; the other is to report the error to the user, prompting the user of possible problems, such as the query conditions entered may be too vague or there are no matches, and suggesting that the user re-enter or adjust the search conditions.

[0156] In the specific implementation scenario of the present invention, suppose that the financial module on the GSCloud enterprise digital cloud platform is being used, and the user wants to find the latest sales records through fuzzy search. The user enters the keyword "sales record" and sets dimension parameters, such as date range, amount range, etc.

[0157] Receive keywords and dimension parameters entered by the user: The user enters "sales records" as the search term, and provides a date range (August 1, 2024 to August 31, 2024) and an amount range (10,000 yuan to 50,000 yuan) as dimension parameters.

[0158] Analyze dimension parameters and determine priorities: The system analyzes these dimension parameters and determines priorities, such as first filtering by date range and then further narrowing the range by amount range.

[0159] Generate candidate result set: Based on these dimension parameters, the system starts searching in the database, uses the FuzzyWuzzy algorithm to match data entries containing "sales records", and generates a candidate result set.

[0160] Reorder the candidate result set: Based on the dimension information of date and amount, the system reorders the candidate result set to obtain the final result set.

[0161] Dimension check: The system checks whether the final result set meets the preset dimension conditions. For example, all records should be within the specified date range, and the sales amount should be within the user-defined interval. If there are records in the result set that do not meet the conditions, such as the date of a record is outside the user-specified range, then this check will fail.

[0162] Pass check: If all records meet the dimension criteria, the system proceeds to the next step.

[0163] Failed the check: If there are records that do not meet the conditions, the system will not return these records. At this time, the system has two options:

[0164] Re-execute the matching process: The system may try to search again, and this time it may adjust the search strategy, such as relaxing the matching conditions or re-analyzing the dimension parameters.

[0165] Error reporting: The system may also report errors directly to the user, indicating that no records matching the criteria were found, and suggesting that the user re-enter the search criteria.

[0166] Return data: If the final result set passes all dimension checks, the system will return the data to the user in sorted order.

[0167] Return error information: If any problems are found during the search process, the system will return corresponding error information to prompt the user of the problem or suggest re-entering the search criteria.

[0168] It should be noted that more detailed dimension checking rules can be developed based on the above solution, allowing users to customize more dimension parameters. For example, in addition to the basic time range and amount range, geographic location, product category and other dimensions can be added. This can make the search more accurate and reduce the possibility of false positives.

[0169] Furthermore, the machine learning model can automatically learn the user's historical search behavior and preferences, thereby dynamically adjusting the criteria for dimension inspection. For example, if the system detects that a user frequently searches for a certain type of information, the relevance score of this type of information can be automatically improved.

[0170] Furthermore, natural language processing (NLP) technology can be added to the fuzzy search process to enhance the system's contextual understanding capabilities, so that the search can better understand the user's true intentions. For example, when a user enters "recent orders", the system can determine from the context whether "recent" refers to today, this week, or this month, and adjust the search scope accordingly.

[0171] Furthermore, a feedback loop mechanism can be established. When users are dissatisfied with the search results, they can easily provide feedback to the system. The system continuously optimizes the matching algorithm and dimension checking rules based on the feedback information to improve the search quality.

[0172] In some embodiments of the present application, receiving keywords and dimension parameters input by a user, analyzing the dimension parameters, and determining priorities are specifically as follows:

[0173] Accept the search character set, dimension information and dimension order input by the user, determine the importance of each dimension and the role of each dimension in the matching process according to the dimension information, and sort the dimensions according to priority according to the dimension order.

[0174] The specific steps of receiving keywords and dimension parameters input by users, analyzing the dimension parameters, and determining priorities are as follows:

[0175] Receive keywords input by users:

[0176] The user enters a search character set through the interface, for example, enters "financial report" as a search keyword.

[0177] The system receives this search character set and stores it as the basis for subsequent search matches.

[0178] Receive dimension information:

[0179] The user simultaneously inputs or selects dimension information, which may include but is not limited to similarity, frequency of use, timestamp, etc.

[0180] For example, a user may specify a similarity threshold of 80%, which means that search results must be at least 80% similar; a usage frequency of high means that frequently mentioned content is displayed first; and a timestamp of the last week means that only updates within the last week are considered.

[0181] Receive dimension order:

[0182] The user specifies the dimension order, such as similarity > usage frequency > timestamp.

[0183] This means that the system considers similarity first, frequency of use second, and timestamp last during the matching process.

[0184] Analysis dimension parameters:

[0185] The system analyzes the incoming dimension parameters and determines the importance of each dimension and their role in the matching process.

[0186] For example, the system recognizes that the user wants the matching results to reach at least 80% similarity, and prioritizes documents that have been cited more times in the past year, while only considering updates within the last month.

[0187] Prioritize:

[0188] Based on the dimension order, the system sorts the dimensions by priority.

[0189] For example, similarity takes precedence over frequency of use, and in the case of equal similarity, documents with higher frequency of use take precedence, and finally timestamps are considered.

[0190] The specific operations are as follows:

[0191] Accept search character set:

[0192] The user enters the search keyword "financial report" and the system stores it as a search character set.

[0193] Receive dimension information:

[0194] The user enters or selects dimension information through the interface, for example:

[0195] Similarity: At least 80% similarity.

[0196] Frequency of use: Documents that have been cited the most times in the past year.

[0197] Timestamp: Updated within the last month.

[0198] Receive dimension order:

[0199] The user specifies the dimension order through the interface, for example: similarity > usage frequency > timestamp.

[0200] Analysis dimension parameters:

[0201] The system analyzes the incoming dimension parameters:

[0202] Similarity: The system recognizes that users want matching results to be at least 80% similar.

[0203] Frequency of use: The system recognizes that the user wants to prioritize documents that have been cited more frequently in the past year.

[0204] Timestamp: The system recognizes that the user wishes to consider only updates within the last month.

[0205] Prioritize:

[0206] Based on the dimension order, the system sorts the dimensions by priority:

[0207] Similarity takes precedence over frequency of use.

[0208] In case of equal similarity, documents with higher frequency of use are given priority.

[0209] Finally consider the timestamp.

[0210] Through the above steps, the system can accurately receive the user's search request and determine the matching rules in the search process according to the user's specific needs and preferences, thereby providing search results that better meet the user's expectations.

[0211] In some embodiments of the present application, the keywords are matched with the database according to the priority to generate a candidate result set, and the candidate result set is reordered according to the dimension parameter to obtain a final result set, specifically:

[0212] The search character set input by the user is split into multiple parts. For each split part, the FuzzyWuzzy algorithm is used to match the target character set to generate a series of result sets; all result sets are merged and duplicates are removed to form a preliminary candidate result set; according to the previously determined dimension information, the candidate result sets are reordered to obtain the final result set.

[0213] Split the search character set: First, the system splits the search character set entered by the user into multiple independent parts. For example, if the user enters "financial report 2021 year", the phrase may be split into "financial report", "2021" and "year".

[0214] Matching using the FuzzyWuzzy algorithm: Next, for each split part, the system uses the FuzzyWuzzy algorithm to match the target character set in the database. The FuzzyWuzzy algorithm uses a variety of methods to evaluate the similarity between two strings, including but not limited to the Levenshtein distance algorithm and Jaccard similarity. This means that even if the search term entered is not exactly the same as the entry in the database, as long as it is similar enough, a match can be found.

[0215] Generate result set: Each part of the match generates a series of results, which represent the set of entries in the database that are closest to that part. For example, "financial report" may match "2021 financial report", "financial statement", etc.

[0216] Merge result sets: All partial matches are merged into one large result set. This process ensures that no duplicate result entries are included in the final set. If "Financial Report" and "2021" each match the same entry, then this entry will only appear once in the merged result set.

[0217] Forming a candidate result set: After removing duplicates, a preliminary candidate result set is formed.

[0218] Sorting by dimensional information: Finally, the candidate result set is re-sorted based on the previously determined dimensional information (e.g., similarity, frequency of use, timestamp, etc.). The dimensional information determines how to prioritize the results to ensure that the most relevant items are ranked first. For example, if the user sets "similarity" as the most important dimension, then those items that are most similar to the search term will be ranked first.

[0219] Get the final result set: After the above steps, the system will get a final result set that has been sorted by priority, and this result set will be returned to the user.

[0220] This process makes fuzzy search more intelligent and efficient, not only able to handle complex query requests, but also ensuring the relevance and accuracy of search results while taking into account the personalized needs of users.

[0221] In some embodiments of the present application, a dimensionality check is performed on the final result set. If it passes, the data is returned. If it fails, the matching process is re-executed or an error is reported. Specifically, the correctness and relevance of the matching results are checked according to the passed dimensional parameters; if the result is correct, the data is prepared to be returned; if the result is incorrect, the similarity is not high enough or is irrelevant to the search content, the matching process needs to be re-executed or an error message is returned.

[0222] The process of dimension checking the final result set is as follows:

[0223] Dimension check: The system checks the correctness and relevance of the matching results based on the dimension parameters entered by the user. Dimension parameters refer to the importance and priority settings used to guide the matching process during the search process, such as similarity threshold, frequency of use, etc.

[0224] Result correctness verification: If the result meets expectations, that is, the result is correct and highly relevant to the search content, the system is ready to return the final result set to the user. The correctness of the result here not only means that the result itself is correct, but also means that the result is highly matched with the search content entered by the user.

[0225] Similarity and relevance evaluation: If the similarity of the matching results is not high enough or is irrelevant to the search content, that is, the matching results do not match the user's search intent, the system needs to take further action. In this case, there are usually two ways to deal with it:

[0226] Re-execute the matching process: The system will restart the search and matching process and may adjust the matching algorithm or parameter settings to obtain better matching results.

[0227] Return error information: If the system determines that it is not possible to obtain better results by trying again, or if other problems are found during the search process (such as input errors, system errors, etc.), the system will return the corresponding error information to the user. The error information will include a description of the problem and possible solutions or suggestions, such as suggesting that the user re-enter the search criteria.

[0228] This mechanism ensures that the data returned to users is both accurate and relevant, while also providing methods for handling abnormal situations, improving user experience and system reliability.

[0229] In some embodiments of the present application, returning data specifically includes returning the sorted final result set to the user.

[0230] Final result set: After completing all matching and sorting steps, the system will obtain a final result set. This result set contains all the items that meet the user's search criteria and have been checked in multiple dimensions.

[0231] Sorting: Based on the dimension parameters defined by the user, the system will sort the final result set. The purpose of sorting is to display the most relevant and user-friendly items to the user first. For example, if the user sets similarity as the highest priority, the system will sort by similarity first; if the similarities are the same, it will sort by frequency of use; and so on.

[0232] Return to user: The system returns the sorted result set to the user. This means that the user will see a prioritized list of results, with the most relevant items at the top of the list.

[0233] The specific steps are as follows:

[0234] The formation of the final result set:

[0235] After a series of matching and screening, the system generates a candidate result set.

[0236] According to the dimension parameters specified by the user, the candidate result set is sorted to obtain the final result set.

[0237] Sorting process:

[0238] The system sorts the candidate result set based on the dimension parameters entered by the user, such as similarity, usage frequency, timestamp, etc.

[0239] For example, if similarity is the most important dimension, the system will first sort by similarity score; if the similarity scores are the same, then sort by frequency; if the frequency is also the same, then sort by timestamp.

[0240] Return to the user:

[0241] The system returns the sorted final result set to the user, who can see a list arranged by priority.

[0242] Users can immediately view the most relevant items, allowing them to quickly find the search results they need.

[0243] For example, suppose a user enters the keyword "financial report" and sets the similarity to 80%, the frequency of use to high, and the timestamp to the last month. After matching and sorting, the system generates a final result set containing the following entries:

[0244] Item A: Similarity 90%, high frequency of use, timestamp is within the last week.

[0245] Item B: Similarity 85%, medium frequency of use, timestamp is the last two weeks.

[0246] Item C: Similarity 80%, high frequency of use, timestamp is the last month.

[0247] The system first sorts by similarity, then by frequency of use if the similarities are the same, and finally by timestamp. Finally, the system ranks item A first, item C second, and item B third, and returns this sorted result set to the user.

[0248] This way, users see an optimized, prioritized list of search results, making it easier to find the most relevant information.

[0249] In some embodiments of the present application, the returning of error information is specifically: if any problem is found during the search process, a corresponding error message is returned to prompt the user of the problem or suggest re-entering the search conditions.

[0250] In some embodiments of the present application, a FuzzyWuzzy algorithm is used to evaluate the similarity between two strings, and the FuzzyWuzzy algorithm includes an edit distance algorithm and Jaccard similarity.

[0251] The FuzzyWuzzy algorithm is a collection of methods for evaluating the similarity between two strings. It measures the similarity between two strings through different algorithms and techniques. In the fuzzy search matching scheme, the FuzzyWuzzy algorithm can handle non-standard inputs such as spelling errors, synonyms, abbreviations, etc., and can evaluate the similarity between strings through various means based on the search conditions entered by the user. The following are several commonly used evaluation methods:

[0252] Edit distance algorithm (Levenshtein distance): This is a method to measure the degree of difference between two strings. It calculates the minimum number of single-character editing operations required to convert one string into another, which includes inserting, deleting, or replacing a character. The smaller the edit distance, the more similar the two strings are.

[0253] Jaccard similarity: The Jaccard similarity coefficient is used to compare the similarity between finite sample sets. For two strings, you can first convert them into sets consisting of characters or character sequences, and then calculate the ratio of the number of elements in the intersection of the two sets to the number of elements in the union. The higher the ratio, the higher the similarity of the two strings.

[0254] In addition to these two methods, FuzzyWuzzy also supports many other matching methods, including but not limited to:

[0255] Simple match: A simple string comparison, usually used to quickly determine whether two strings are exactly the same.

[0256] Non-exact match: Partial matches are allowed, meaning that the strings do not have to be exactly the same to be considered similar.

[0257] Ignore order matching: Ignore the order in which characters appear when comparing, and only care about whether the characters exist in both strings.

[0258] Deduplication subset matching: Remove duplicate characters or character sequences during the matching process to avoid the impact of duplicate counting.

[0259] The advantage of the FuzzyWuzzy algorithm is that it can flexibly select different matching strategies to adapt to different application scenarios and needs. For example, when processing large-scale data, FuzzyWuzzy can still maintain a relatively fast matching speed and improve overall efficiency; at the same time, it can select the appropriate matching method according to actual needs, thereby improving the accuracy and efficiency of matching.

[0260] In summary, the FuzzyWuzzy algorithm provides a comprehensive fuzzy search matching solution by combining multiple evaluation methods, so that the system can more accurately understand and meet user intentions when processing complex search requests.

[0261] Example 2

[0262] The character sets passed in include two, namely character sets and character sets , the dimension passed in is , the order of the incoming dimensions is .

[0263] in, It is the search character set that stores the search content. It is the character set that stores the search matching target. is the corresponding dimension related information, is the order of the dimensions passed in, which determines The order and relevance priority of the dimensions. and Configurable, through which you can obtain different matching results.

[0264] To perform dimensional analysis of data, it is necessary to combine the incoming dimensional parameters and analyze them in the order of the incoming dimensions. The order and relevance priority of the dimensions are determined, and then the priority and order of the matching algorithm are determined by the dimensions.

[0265] The specific operation can be simplified as follows:

[0266]

[0267] in is based on The dimension of the ordinal numbering.

[0268] Search content character set By splitting, the search content is obtained separately , , … For the target character set , get the result set through FuzzyWuzzy , , … . For the obtained result set, merge and remove duplicates to obtain the target result set .

[0269] The details are as follows:

[0270] Get the result set After that, through the dimension (Specific dimensions are , , … ), re-sort the dimensions and obtain the final result set

[0271] The details are as follows:

[0272]

[0273] Check the dimension according to the passed dimension parameter. If it passes, prepare to return data; if it is wrong (similar association has problems, and The association does not match the content, If the character set content is unreasonable, etc.), it is necessary to re-iterate the sort or return an error.

[0274] Will return in a determined order, or return an error message.

[0275] Example 3

[0276] A computer program product containing instructions, when running on a device, enables the device to execute the steps in a general data fuzzy search and matching method based on a FuzzyWuzzy algorithm.

[0277] Example 4

[0278] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of a general data fuzzy search and matching method based on a FuzzyWuzzy algorithm.

[0279] Example 5

[0280] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor. When the processor executes the program, steps in a general data fuzzy search and matching method based on a FuzzyWuzzy algorithm are implemented.

[0281] Anything not described in this application can be achieved by adopting or drawing on existing technologies.

[0282] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0283] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A general data fuzzy search and matching method based on FuzzyWuzzy algorithm, characterized in that: include: Receive keywords and dimension parameters input by the user, analyze the dimension parameters, and determine the priority, specifically: accept the search character set, dimension information, and dimension order input by the user, determine the importance of each dimension and the role of each dimension in the matching process according to the dimension information, and sort the dimension information according to the priority according to the dimension order; The dimension information includes similarity threshold, usage frequency and timestamp; According to the priority, the keywords are matched with the database to generate a candidate result set, and according to the dimension parameter, the candidate result set is reordered to obtain a final result set, specifically: the search character set input by the user is split into multiple parts, and for each of the split parts, the FuzzyWuzzy algorithm is used to match the target character set to generate a series of result sets; all the result sets are merged and duplicates are removed to form a preliminary candidate result set; the candidate result set is reordered according to the priority order of the dimension parameter previously determined to obtain a final result set; Performing a dimension check on the final result set, returning data if it passes, and re-executing the matching process or reporting an error if it fails, specifically: checking the correctness and relevance of the matching result according to the dimension parameters passed in; If the result is correct, the data is returned; if the result is incorrect, the similarity is not high enough, or it is irrelevant to the search content, the matching process needs to be re-executed or an error message is returned; After completing all matching and sorting steps, the system will obtain a final result set. According to the dimension parameters defined by the user, the system will sort the final result set and return the sorted result set to the user.

2. The method according to claim 1, characterized in that The dimension check is performed on the final result set. If it passes, the data is returned. If it fails, the matching process is re-executed or an error is reported. Specifically, the correctness and relevance of the matching result are checked according to the dimension parameters passed in. If the result is correct, the data is prepared to be returned; if the result is incorrect, the similarity is not high enough or it is irrelevant to the search content, the matching process needs to be re-executed or an error message needs to be returned.

3. The method according to claim 2, characterized in that The returned data specifically includes: returning the sorted final result set to the user.

4. The method according to claim 2, characterized in that: The returning error information specifically means: if any problem is found during the search process, a corresponding error message is returned to prompt the user of the problem or suggest re-entering the search conditions.

5. The method according to claim 1, characterized in that: The FuzzyWuzzy algorithm is used to evaluate the similarity between two strings, and the FuzzyWuzzy algorithm includes an edit distance algorithm and a Jaccard similarity.

6. A computer program product comprising instructions, which, when executed on a device, is characterized in that: The device is enabled to execute the steps in the general data fuzzy search and matching method based on the FuzzyWuzzy algorithm as described in any one of claims 1 to 5.

7. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the general data fuzzy search and matching method based on the FuzzyWuzzy algorithm as described in any one of claims 1 to 5 are implemented.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the general data fuzzy search and matching method based on the FuzzyWuzzy algorithm as described in any one of claims 1 to 5 are implemented.

Images