A text retrieval method and device based on multi-link word segmentation

Abstract

This application relates to a text retrieval method and apparatus based on multi-link word segmentation. The method includes: receiving original search terms and classifying them according to their character morphological features to obtain the input category to which the original search terms belong; if the input category to which the original search terms belong is a multi-keyword category, performing word segmentation processing on the original search terms using a first word segmentation link to obtain a first link processing result; if the first link processing result indicates that the first word segmentation link timed out, was abnormal, or invalid for the original search terms, performing word segmentation processing on the original search terms using a second word segmentation link to obtain a second word segmentation link processing result; constructing search query conditions and performing a search in a target document database according to the search query conditions to obtain the search results corresponding to the original search terms. This application eliminates the segmentation ambiguities generated when a single word segmentation strategy deals with multiple input forms, improving the stability of the retrieval recall level.

Description

Technical Field

[0001] This application relates to the field of document retrieval technology, and in particular to a text retrieval method and apparatus based on multi-link word segmentation. Background Technology

[0002] With the rapid growth of enterprise-level data scale, full-text search technology for multiple business sub-databases such as document repositories, information repositories, and research report repositories has become one of the core components of information systems. In full-text search scenarios, search engines typically have a built-in tokenizer to segment the user-input search terms into several terms, and then perform inverted index matching in the document repository based on these terms. In traditional solutions, the system directly submits the user-input raw search terms to the search engine's tokenizer for processing. The tokenizer segments the search terms based on its built-in dictionary and rules, and then uses the segmented terms to construct query conditions and execute the search.

[0003] The aforementioned single-segmentation-link processing method presents a technical problem when facing inputs of different forms: because the segmentation strategy of the segmenter is fixed, for complex search terms containing multiple semantic fragments, the terms output by the single-segmentation-link are prone to semantic boundary recognition errors; while for short words or precise phrases, the single-segmentation-link tends to over-segment them into finer-grained terms. This segmentation ambiguity caused by the single-segmentation-link handling of multiple input forms directly affects the determination of document hits and the number of hit documents during the inverted index matching process, thus leading to unstable output results of the retrieval system at the recall level. Summary of the Invention

[0004] Therefore, it is necessary to provide a text retrieval method and apparatus based on multi-link word segmentation that can select word segmentation links according to the input form to improve the stability of retrieval recall, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a text retrieval method based on multi-link word segmentation, including:

[0006] Receive the original search terms and classify them according to their character morphological features to obtain the input category to which the original search terms belong;

[0007] When the input category to which the original search term belongs is a multi-keyword category, the original search term is segmented using the first segmentation link to obtain the first link processing result;

[0008] If the first segmentation link's processing result indicates that the first segmentation link's segmentation processing of the original search term timed out, was abnormal, or was invalid, then the second segmentation link is used to segment the original search term to obtain the second segmentation link's processing result; wherein, the segmentation processing quality of the first segmentation link is higher than that of the second segmentation link;

[0009] Based on the results of the first or second word segmentation process, construct search query conditions;

[0010] Based on the search query conditions, a search and matching process is performed in the target document library to obtain the search results corresponding to the original search terms.

[0011] Secondly, this application also provides a text retrieval device based on multi-link word segmentation, comprising:

[0012] The receiving module is used to receive the original search terms and classify the original search terms according to the character morphological features of the original search terms to obtain the input category to which the original search terms belong;

[0013] The first word segmentation processing module is used to segment the original search term using a first word segmentation link when the input category to which the original search term belongs is a multi-keyword category, and to obtain the first link processing result;

[0014] The second word segmentation processing module is used to perform word segmentation processing on the original search term using the second word segmentation link when the processing result of the first link indicates that the word segmentation processing of the first word segmentation link for the original search term has timed out, is abnormal, or is invalid, so as to obtain the processing result of the second word segmentation link; wherein, the word segmentation processing quality of the first word segmentation link is higher than that of the second word segmentation link.

[0015] The module is used to construct search query conditions based on the results of the first or second word segmentation link processing.

[0016] The retrieval module is used to perform a retrieval match in the target document library according to the retrieval query conditions, and obtain the retrieval results corresponding to the original retrieval terms.

[0017] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps described in the first aspect.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps described in the first aspect.

[0019] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps described in the first aspect.

[0020] The aforementioned text retrieval method and apparatus based on multi-link word segmentation first classifies the search terms according to their character morphological features, identifying the input category to which the search term belongs. This allows different forms of search terms to be treated differently in subsequent processing, avoiding segmentation ambiguities caused by sending all forms of input to the same word segmentation link. Second, when a search term is classified into multiple keyword categories, the first word segmentation link with higher processing quality is prioritized for word segmentation. Furthermore, if the first word segmentation link times out, it immediately switches to the second word segmentation link for word segmentation processing. The retrieval query conditions are then constructed based on the results of the second word segmentation link to perform retrieval matching. This ensures that even if the first word segmentation link malfunctions, the retrieval process can continue and obtain usable word segmentation results to complete the retrieval, preventing the entire retrieval request from failing due to the malfunction of a single link. This application enables different forms of input to adapt to corresponding word segmentation strategies and has automatic recovery capabilities when a high-quality word segmentation link is unavailable, thereby eliminating segmentation ambiguities caused by a single word segmentation strategy dealing with multiple input forms and improving the stability of retrieval recall. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an application environment diagram of a text retrieval method based on multi-link word segmentation in one embodiment;

[0023] Figure 2 This is a flowchart illustrating a text retrieval method based on multi-link word segmentation in one embodiment;

[0024] Figure 3 This is a schematic diagram of the process for generating search query conditions in one embodiment;

[0025] Figure 4 This is an overall interaction sequence diagram of a text retrieval method based on multi-link word segmentation in one embodiment;

[0026] Figure 5 This is a structural block diagram of a text retrieval device based on multi-link word segmentation in one embodiment;

[0027] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0030] This application provides a text retrieval method based on multi-link word segmentation, which can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. After receiving the original search terms submitted by terminal 102, server 104 classifies them according to their character morphological characteristics to identify the input category to which the search terms belong. When a search term is classified into multiple keyword categories, server 104 prioritizes calling the first segmentation link with higher segmentation quality for segmentation processing. If the first segmentation link times out, is abnormal, or invalid, it switches to the second segmentation link for segmentation processing. Then, based on the segmentation results, it constructs search query conditions and finally performs a search matching in the target document database based on these query conditions, returning the search results to terminal 102. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0031] In an exemplary embodiment, as shown in Figure 2 , a text retrieval method based on multi-link word segmentation is provided, and the method is described by taking its application to the server in Figure 1 as an example, which includes the following Step 201 to Step 205:

[0032] Step 201: Receive an original search term, and classify the original search term according to the character morphological features of the original search term to obtain the input category to which the original search term belongs.

[0033] In some embodiments, the server first receives the original search term input by the user through the search interface. The original search term is the to-be-queried text submitted by the user in the search box, and its form may include, but is not limited to, short Chinese words, long Chinese sentences, continuous English words, English phrases with spaces, Chinese-English mixed strings, exact phrases in quotation marks, stock codes or numeric strings, etc. After receiving the original search term, the server does not immediately send the search term to the search engine for retrieval, but first submits the search term to the keyword analysis module inside the server for standardized preprocessing.

[0034] It can be understood that there are significant differences in the requirements for word segmentation strategies for different types of input forms. For example, if a single Chinese word "semiconductor" is over-segmented into three single characters "semiconductor", it will lead to divergent recall results and reduced exact matching capability; while a space-free long string such as "domestic substitution of semiconductor equipment" is prone to being split into semantically fragmented pieces without semantic recognition. Therefore, before word segmentation processing, it is necessary to pre-classify the input samples to distinguish which inputs are suitable for direct matching and which inputs need to enter the complex word segmentation link.

[0035] In practical applications, the input pre-classification module of the server completes the classification according to character morphological features such as the number of characters, space distribution, quotation mark wrapping, continuous English features, and whether a stock code-like pattern is included. In one embodiment, the character morphological features include at least one of the following judgment dimensions: whether the original search term contains spaces or separators, whether the original search term contains quotation marks, whether the original search term consists only of a single Chinese word or a single continuous English word, whether the original search term is in the stock code format, and whether the original search term is a space-free continuous Chinese character string with a length exceeding a preset threshold. The pre-classification module divides the original search term into at least two input categories according to the detection results of the above dimensions, including a single keyword category and a multi-keyword category. In this embodiment, the case of the multi-keyword category is described first, and the case of the single keyword category will be described later.

[0036] Step 202: When the input category to which the original search term belongs is the multi-keyword category, perform word segmentation processing on the original search term by adopting a first word segmentation link, so as to obtain a first link processing result.

[0037] In some embodiments, when the input pre-classification module determines that the input category to which the original search term belongs is a multi-keyword category, the server sends the original search term to the first word segmentation link for word segmentation processing. The first word segmentation link is a word segmentation link with higher word segmentation processing quality than the second word segmentation link. Its higher processing quality is reflected in the fact that, for complex multi-keyword inputs, the first word segmentation link can output more accurate semantic segmentation results, more reasonable word boundary division, and more accurate key word recognition capabilities, thereby better distinguishing the core concepts and modifying components in the user's query.

[0038] In one embodiment, the first word segmentation link is an AI (Artificial Intelligence) word segmentation link, which intelligently segments the original search terms based on a large language model or a deep learning semantic understanding model. The AI ​​word segmentation engine outputs a sequence of terms based on the contextual semantic information of the input text, and can simultaneously provide candidate flags indicating whether each term should be included in the key matching set. Specifically, the AI ​​word segmentation engine receives the original search terms and segmentation suggestions, performs semantic segmentation on the search terms using a semantic understanding model, and outputs an AI term list and key word suggestions. The semantic model used by the AI ​​word segmentation engine can be a large language model fine-tuned with domain corpus, or a specially trained word segmentation and sequence labeling model. Its core advantage lies in its ability to understand the semantic boundaries and topic levels in complex multi-keyword combinations. For example, it can correctly identify the two core concepts "semiconductor equipment" and "domestic substitution" in "domestic substitution of semiconductor equipment," rather than splitting the input into four independent segments: "semiconductor," "equipment," "domestic," and "substitution."

[0039] Understandably, while AI-powered word segmentation offers high segmentation quality, it consumes significant computational resources, has a relatively long response time, and is susceptible to timeouts or anomalies due to network conditions and model service load. Therefore, when using AI-powered word segmentation, its execution process should be monitored, and a reasonable circuit breaker mechanism should be implemented to ensure the overall availability of the retrieval service. The first-stage processing results obtained by the server in this step include, but are not limited to: the word list returned by the AI ​​word segmentation engine, part-of-speech tagging for each word, key matching candidate markers, and the time taken for this segmentation. After receiving the first-stage processing results, the server's keyword analysis module performs validity checks, including but not limited to verifying whether the returned results are empty, whether the word structure is complete, and whether the response format is parsable.

[0040] Step 203: If the first segmentation link's processing result indicates that the first segmentation link's processing of the original search term has timed out, is abnormal, or is invalid, the second segmentation link is used to segment the original search term to obtain the second segmentation link's processing result; wherein, the segmentation quality of the first segmentation link is higher than that of the second segmentation link.

[0041] In some embodiments, the server continuously monitors the call time of the first word segmentation link while or after invoking it, and compares the actual time with a preset timeout threshold. If the actual time exceeds the preset threshold, the server determines that the first word segmentation link has timed out, and the processing result of the first link indicates that the word segmentation link is unavailable. After determining the timeout, the server immediately triggers a circuit breaker mechanism, interrupting or ignoring the waiting state of the current AI word segmentation link, and switching to the second word segmentation link to perform word segmentation processing on the original search terms.

[0042] The second word segmentation link is a link with lower processing quality but higher stability compared to the first word segmentation link. In this invention, the second word segmentation link is preferably a search engine word segmentation link, that is, it uses the search engine's built-in search word segmentation analysis interface to perform word segmentation processing on the original search terms. The search engine word segmentation link outputs word segmentation results with word position information based on dictionary word segmenters or rule word segmenters. It has stable processing speed, low resource consumption, and high engineering availability, making it suitable as a backup link for AI word segmentation links.

[0043] It should be noted that the conditions for triggering the switch from the first word segmentation link to the second word segmentation link are not limited to timeout. In some embodiments, in addition to timeout, the server also switches to the second word segmentation link when the processing result of the first link meets any of the following unusable conditions, namely, exceptions, invalidity, etc.: First, the AI ​​word segmentation interface returns an abnormal response (such as HTTP 5xx error, network timeout, etc.); second, the AI ​​word segmentation interface response is empty or the returned result cannot be parsed; third, although the AI ​​word segmentation result returns successfully, the term structure does not meet the validity requirements. In one embodiment, the server comprehensively judges the validity of the first word segmentation link. Only when the AI ​​word segmentation result simultaneously meets the three conditions of "no timeout", "non-empty response", and "complete term structure" will the server consider the processing result of the first link usable and adopt it; otherwise, the server switches to the second word segmentation link.

[0044] For example, continuing with the previous example of "semiconductor equipment, domestic substitution", if the AI ​​word segmentation engine fails to return a result within 3 seconds due to excessive model service load, the server's keyword analysis module triggers a circuit breaker when the timer reaches 3 seconds, abandons waiting for the AI ​​word segmentation result, and immediately calls the search engine's word segmentation analysis interface to perform word segmentation on "semiconductor equipment domestic substitution". The search engine's word segmentation unit returns word segmentation results with word position information based on the search analyzer, such as returning the word sequence ["semiconductor", "equipment", "domestic", "substitution"] and their respective word position information. The server's keyword analysis module records this result as the result of the second word segmentation link processing.

[0045] Step 204: Based on the results of the first or second word segmentation link processing, construct the search query conditions.

[0046] In some embodiments, after obtaining the first or second word segmentation link processing result, the server does not directly send the raw segmented strings from the first or second word segmentation link processing result to the search engine. Instead, it first structures and organizes the first or second word segmentation link processing result, and then constructs search query conditions based on the organized standard term set. These search query conditions are a query description structure that the search engine can recognize and execute. The search query conditions at least include the term content to be matched, the matching fields corresponding to each term, and the matching weight of each term in each field.

[0047] It should be noted that the core of this step lies in converting the raw word segmentation results from the first or second word segmentation link into formatted query conditions that can be executed by the retrieval cluster. This conversion process involves classifying, labeling, and assigning weights to each word in the first or second word segmentation link results, so that the matching contributions of different terms in different document fields can be distinguished during subsequent retrieval execution.

[0048] Step 205: Perform a search and match in the target document library according to the search query conditions to obtain the search results corresponding to the original search terms.

[0049] The server submits the search query to the search cluster, which then performs a search and matching process in the target document database. Search matching refers to the search cluster calculating the relevance score of each document in the target document database to the original search terms based on the matching rules and weights of each term in the search query in each field, and returning a set of candidate documents according to the scores.

[0050] This application first classifies search terms based on their character morphological features, identifying the input category to which each search term belongs. This allows different search terms to be treated differently in subsequent processing, avoiding segmentation ambiguities caused by sending all inputs of the same form to the same segmentation link. Second, when a search term is classified into multiple keyword categories, the first segmentation link with higher segmentation quality is prioritized. Furthermore, if the first segmentation link times out, the process immediately switches to the second segmentation link for segmentation. The search query conditions are then constructed based on the results of the second segmentation link to perform search matching. This ensures that even if the first segmentation link malfunctions, the search process can continue and obtain usable segmentation results to complete the search, preventing the entire search request from failing due to a single link's failure. This application enables different input forms to adapt to corresponding segmentation strategies and has automatic recovery capabilities when a high-quality segmentation link is unavailable, thereby eliminating segmentation ambiguities caused by a single segmentation strategy dealing with multiple input forms and improving the stability of the search recall level.

[0051] Figure 3 For a schematic diagram of the query condition generation process in one embodiment, please refer to [link / reference]. Figure 3 In one optional embodiment, based on the result of the first word segmentation link processing or the result of the second word segmentation link processing, retrieval query conditions are constructed, including:

[0052] Step 301: Obtain at least one field from the target document library that participates in the retrieval and matching. The at least one field includes a first type of field and a second type of field. The first type of field contains the title information of the target document library, and the second type of field contains the body information of the target document library.

[0053] The server retrieves the field configuration information for retrieval matching from the target document library. The target document library is the collection of documents to be retrieved, which may include one or more business sub-libraries, such as research report sub-libraries, news sub-libraries, announcement sub-libraries, knowledge base sub-libraries, etc. Documents in each business sub-library contain multiple structured fields, and different fields contribute differently to the relevance of the retrieval. Therefore, it is necessary to clarify which fields participate in the matching and the importance of each field before constructing the query conditions.

[0054] At least one field includes both a first-class field and a second-class field. The first-class field contains title information from the target document library, specifically including summary text fields such as the original document title, translated title, and subtitle. Title information is usually extracted by the document author or editor and centrally reflects the core theme and main content of the document; therefore, term matching in title fields often has higher relevance indicative significance. The second-class field contains body text information from the target document library, specifically including descriptive text fields such as document summary, translated summary, body text, and paragraph text. Body text information is a complete expansion and detailed description of the document content, with a much larger text volume than the title. Although the importance of matching individual terms in the body text is relatively lower than that in the title, body text matching provides a richer source of recall.

[0055] Step 302: Determine the segmentation type of each word in the first link processing result or the second segmentation link processing result; the segmentation type includes key word type and non-key word type.

[0056] After obtaining the processing results of the first or second word segmentation link, the server determines the segmentation type for each word in the first or second word segmentation link processing results. The segmentation type is determined based on whether the word carries a key word tag, which will be explained in detail later.

[0057] It should be noted that word segmentation is the main word level obtained after processing the results of the first or second word segmentation link through word position grouping and main word-synonym mapping. That is, the server determines the word segmentation type for the main word in each word position group, and synonyms in the same word position group inherit the word segmentation type of the main word.

[0058] The sources of key word tags can be varied. In one embodiment, when the processing results of the first or second word segmentation link come from the search engine's word segmentation link, the key word tags are primarily determined by the matching results of the business key word thesaurus. The server pre-builds or maintains a business key word thesaurus, which contains core terms, popular topic words, high-frequency query words, and other words with high retrieval value within the domain. For each main word in the processing results of the first or second word segmentation link, the server queries whether the main word matches the business key word thesaurus; if it matches, a key word tag is added to the main word, and its word segmentation type is determined to be a key word type; if it does not match, its word segmentation type is determined to be a non-key word type.

[0059] Exemplarily, it is assumed that the main terms in the processing result of the first word segmentation link or the processing result of the second word segmentation link include "semiconductor equipment", "domestic substitution", "of", and "research", wherein "of" is filtered and removed by matching the stop word list; "semiconductor equipment" matches the business key word thesaurus, is attached with a key word tag, and its word segmentation type is determined as the key word type; "domestic substitution" does not match the business key word thesaurus and does not carry the AI key matching candidate flag, and its word segmentation type is determined as the non-key word type; "research" also does not match the key word thesaurus, and its word segmentation type is determined as the non-key word type.

[0060] Step 303: Determine the matching weight of each segmented word in each field according to the word segmentation type. Wherein, for the same field, the matching weight of the segmented word corresponding to the key word type is greater than the matching weight of the segmented word corresponding to the non-key word type; for the same segmented word, the matching weight of the segmented word in the first-type field is greater than the matching weight of the segmented word in the second-type field.

[0061] After determining the word segmentation type of each segmented word in the processing result of the first word segmentation link or the processing result of the second word segmentation link, the server assigns a matching weight to each segmented word in each field according to the word segmentation type and the field type (the first-type field or the second-type field). The matching weight is used to represent the contribution degree of the segmented word to the document relevance score when the segmented word is hit in the corresponding field in the subsequent retrieval and matching process.

[0062] Specifically, the allocation of matching weights follows two core rules. The first rule is that for the same field, the matching weight of the segmented word corresponding to the key word type is greater than the matching weight of the segmented word corresponding to the non-key word type. That is to say, in the title-type field, the matching weight of the key word is greater than that of the non-key word; in the body-type field, the matching weight of the key word is also greater than that of the non-key word. This rule reflects the core position of key words in retrieval. The terms marked as key words represent the most critical concepts that need to be forcibly matched in the user input, so their matching in any field should obtain a higher score contribution.

[0063] The second rule is that for the same segmented word, the matching weight of the segmented word in the first-type field is greater than the matching weight of the segmented word in the second-type field. That is to say, no matter the segmented word is a key word or a non-key word, its matching weight in the title-type field is greater than its matching weight in the body-type field. This rule reflects that the title information has higher relevance indication value than the body information, because the document title is usually an accurate summary of the document topic, so the hit of a term in the title has stronger indication significance for relevance than the hit in the body.

[0064] Step 304: Generate a retrieval query condition according to each segmented word and the matching weight of each segmented word in each field.

[0065] The server generates search query conditions based on the segmented content and the corresponding matching weights. These conditions include at least the segmented content to be matched, the matching fields corresponding to each segment, and the matching weight of each segment in each field.

[0066] In one embodiment, for each word segment, the server generates matching clauses for that word in a first-category field and matching clauses for that word in a second-category field, each matching clause carrying a corresponding matching weight value. Then, the server combines all matching clauses into a complete search query condition according to a preset logical combination method (e.g., "OR" logic). In this way, when performing search matching, the retrieval cluster scores the hit rate of each word in each field separately, and then weights and sums them according to the matching weights to obtain the document's relevance score.

[0067] Specifically, in one embodiment, the search query conditions generated by the server also include mandatory matching filters for key words. The mandatory matching filter requires that candidate documents must match at least one key word segment (or its synonym); otherwise, the document will be excluded from the search results. This filter ensures that the returned documents are strongly relevant to the core concept entered by the user, preventing documents that only match non-key words but deviate from the overall topic from being recalled.

[0068] The beneficial effect of this embodiment is that by obtaining at least one field involved in the retrieval and matching from the target document library and classifying it into a first type of field and a second type of field, a field dimension basis is provided for subsequent differentiated weight configuration, enabling title matching and body text matching to play different roles in relevance calculation. By determining the segmentation type of each word in the first or second segmentation link processing result, distinguishing between key word types and non-key word types, core concept words and general terms in the user input play different functions in the retrieval. Key words play the role of topic positioning, while non-key words play the role of auxiliary matching. This allows the document's relevance score to accurately reflect the document's hit rate for terms of different importance in different fields, thus returning results that are more focused on the core topics of interest to the user.

[0069] In one optional embodiment, performing a search matching in the target document library based on the search query conditions to obtain the search results corresponding to the original search terms includes: performing a search matching in the target document library based on the matching weight of each word in each field contained in the search query conditions to determine multiple candidate documents and their respective relevance scores; obtaining the time information of each candidate document, where the time information represents the publication time or update time of the corresponding candidate document; arranging the multiple candidate documents in descending order of relevance score according to their respective relevance scores; arranging the candidate documents with the same relevance score in ascending order of time information to determine the sorting position of each candidate document; and outputting each candidate document according to its respective sorting position to obtain the search results corresponding to the original search terms.

[0070] After constructing the search query criteria, the server submits the query criteria to the search cluster, which then performs search matching in the target document database. Search matching refers to the search cluster calculating the relevance score of each document in the target document database to the original search terms based on the matching weight of each word in the search query criteria in each field, and determining multiple candidate documents and their respective relevance scores based on the matching results.

[0071] In some embodiments, after receiving the search query conditions sent by the server, the retrieval cluster, for each document in the target document library, iterates through the first and second categories of fields of the document, checking whether the word segmentation content contained in the search query conditions exists in each field. Specifically, for any word segmentation in the search query conditions, if the word segmentation appears in the title category field of a document, the retrieval cluster accumulates the corresponding score for the document based on the matching weight of the word segmentation in the title category field; if the word segmentation appears in the body category field of the document, the retrieval cluster accumulates the corresponding score for the document based on the matching weight of the word segmentation in the body category field. By iterating and accumulating the hit status of all words in the search query conditions in each field, the retrieval cluster calculates the relevance score of each document.

[0072] In one embodiment, the retrieval cluster also performs forced matching filtering for key word segmentation types when performing retrieval matching. Specifically, before or simultaneously with calculating the document relevance score, the retrieval cluster first checks whether each document matches at least one key word segmentation type or its synonym. If a document does not match any key word segmentation type, the retrieval cluster excludes the document from the candidate documents list, even if the document matches multiple non-key word segmentation types.

[0073] It should be noted that the above calculation method for relevance scores is illustrative. In practical applications, the retrieval cluster can combine the search engine's built-in relevance scoring algorithm with the aforementioned matching weights for a comprehensive score. Specifically, the retrieval cluster can input the matching weights of each word segment in each field as query enhancement factors into the search engine, which will then calculate the document's relevance score based on term frequency, inverse document frequency, and the enhancement factor. This combined approach takes into account both statistical relevance and business weight configuration, resulting in more accurate ranking results.

[0074] For example, suppose that "semiconductor equipment" is a key term in the search query, with a matching weight of 1000 in the title field and 100 in the body text field; "domestic substitution" is a non-key term, with a matching weight of 100 in the title field and 10 in the body text field. After the search cluster performs a search and match on the target document database, if candidate document A's title contains "semiconductor equipment" and its body text contains "domestic substitution" (a synonym of "domestic substitution"), then this document receives 1000 points from the key term in the title, plus 10 points from the non-key term in the body text, for a relevance score of 1010. If candidate document B's title does not contain "semiconductor equipment," but its body text contains both "semiconductor equipment" and "domestic substitution," then this document receives 100 points from the key term in the body text, plus 10 points from the non-key term in the body text, for a relevance score of 110. Candidate document C's body text only contains "equipment" and "alternative," failing to match the key keyword "semiconductor equipment." Therefore, this document is excluded from the candidate documents because it does not meet the mandatory matching criteria. It is evident that documents with titles matching the key keyword receive significantly higher relevance scores than other documents, thus gaining a higher ranking position.

[0075] Each document in the target document library contains a time attribute field, which records the document's publication date or most recent update date. Before constructing search queries or after the search cluster returns candidate documents, the server retrieves the content of this time attribute field as the candidate document's time information. Specifically, the server can read the publication date or update date of each candidate document from the target document library's metadata. This time information can be a timestamp format accurate to the year, month, day, or even hour, minute, and second, or a simplified format accurate only to the year.

[0076] In one embodiment, when constructing the search query conditions, the server includes the time information field in the list of returned fields, so that the retrieval cluster returns the time information of each candidate document along with the candidate document list. In this way, the server obtains the time information of each candidate document simultaneously with receiving the candidate document list, without requiring additional queries. In another embodiment, after obtaining the candidate document list, the server queries the target document database in batches for the time information of each candidate document based on its unique identifier.

[0077] It should be noted that both publication time and update time are effective indicators for measuring the timeliness of documents. For research reports and announcements, the publication time typically refers to the date the document was first published, accurately reflecting the freshness of the information. For news documents, the publication time refers to the date the information was published. For knowledge base documents, the update time reflects the validity of the document content better than the publication time. In one embodiment, the server prioritizes obtaining the publication time of candidate documents as the time information; if the publication time of a candidate document is empty or invalid, then the update time of that document is obtained as the time information.

[0078] Understandably, the purpose of obtaining the time information of candidate documents is to provide a timeliness dimension for subsequent composite ranking. Simply relying on relevance scores for ranking might result in older documents with lower timeliness but slightly higher relevance being ranked ahead of newer documents with better timeliness but slightly lower relevance, negatively impacting user experience. Therefore, introducing time information as a secondary ranking dimension on top of relevance score ranking can effectively balance the relationship between relevance and timeliness.

[0079] The server performs a sorting operation on the candidate document list returned by the retrieval cluster. The sorting operation uses relevance score as the primary sorting key, arranging the candidate documents in descending order of score. Specifically, the server compares the relevance scores of any two candidate documents, ranking the document with the higher score before the document with the lower score. Through descending sorting, the most relevant document is placed at the beginning of the candidate document list.

[0080] In one embodiment, if two or more candidate documents have the same relevance score, the server temporarily maintains the relative order of these documents and allows subsequent steps to perform a secondary sort based on time information. In other words, relevance score, as the primary ranking dimension, determines the overall ranking range of candidate documents; documents with higher scores are generally ranked before those with lower scores. Only when scores are identical is it necessary to break the ranking tie through other factors. It is understandable that, as relevance score is the main ranking dimension, documents that best match the user's query intent are returned first.

[0081] After completing the initial sorting based on relevance scores, the server arranges the candidate documents with the same relevance scores in order of newest to oldest time information, and determines the sorting position of each candidate document.

[0082] The server iterates through candidate document groups with the same relevance score, and sorts the candidate documents within each group from newest to oldest based on their publication date. Specifically, for candidate documents with the same relevance score, documents with newer publication or update dates are ranked before older documents. Through this secondary sorting, among candidate documents with comparable relevance levels, documents with better timeliness are assigned a higher ranking position.

[0083] For example, suppose there are three candidate documents: Document A has a relevance score of 100 and was published in June 2025; Document B has a relevance score of 100 and was published in December 2024; and Document C has a relevance score of 95 and was published in January 2025. The server first sorts them in descending order of relevance score, with Document A and Document B (both with a score of 100) ranked before Document C (with a score of 95). Between Document A and Document B with the same score, the server sorts them in ascending order of publication date, with Document A (June 2025) ranked before Document B (December 2024). The final ranking is: Document A, Document B, Document C.

[0084] The beneficial effect of this embodiment is that it performs retrieval matching and determines the relevance score of candidate documents based on the matching weight of each word in each field of the retrieval query conditions. This ensures that the calculation of document relevance score accurately reflects the differentiated contribution of terms of different importance when hitting different types of fields, improving the rationality and distinguishability of relevance score. By obtaining the time information of each candidate document, a timeliness dimension is introduced into the sorting process. By first arranging multiple candidate documents according to relevance score from high to low, and then arranging candidate documents with the same relevance score according to time information from newest to oldest, a composite sorting with relevance priority and time secondary is constructed. This ensures that the retrieval results prioritize displaying documents with better timeliness while maintaining topic relevance, balancing retrieval accuracy and users' need for fresh information.

[0085] In an optional embodiment, the method further includes: obtaining the word position information corresponding to each word in the first link processing result or the second word segmentation link processing result; the word position information is used to indicate the word position of the corresponding word in the original search term; dividing the words with the same word position information into the same word position group to obtain at least one word position group; for each word position group, taking the first word in the word position group as the main word, and taking the other words in the word position group other than the first word as synonyms of the main word; determining the key word tag corresponding to each main word according to the main word in each word position group, and attaching the key word tag to the corresponding main word; wherein, the key word tag is used to determine the word segmentation type of each word.

[0086] The server retrieves the word position information corresponding to each word in the first or second word segmentation processing result. The word position information is used to indicate the word position of the corresponding word in the original search term, that is, the start and end positions of the word in the character sequence of the original search term.

[0087] In some embodiments, when outputting segmentation results, the search engine's word segmentation unit simultaneously returns the lexical position information corresponding to each segmented word. Taking the Elasticsearch search engine as an example, after its word segmentation analysis interface segments the input text, the returned segmentation results typically include the start and end offsets of each segmented word in the original text. These two offsets together constitute the lexical position information of that segmented word. The server's keyword analysis module extracts the lexical position information of each segmented word from the results returned by the search engine's word segmentation unit and associates and stores the segmented word content with its lexical position information.

[0088] For example, assuming the original search term is "domestic substitution of semiconductor equipment", the search engine's word segmentation unit performs word segmentation processing on this search term, returning the segmentation results and corresponding word position information as follows: "semiconductor" corresponds to word positions 0-2, "equipment" corresponds to word positions 3-4, "domestic" corresponds to word positions 5-6, and "substitution" corresponds to word positions 7-8. Here, "0-2" indicates that "semiconductor" occupies positions from the 0th to the 2nd character in the original search term, "3-4" indicates that "equipment" occupies positions from the 3rd to the 4th character, and so on. The server's keyword analysis module obtains this word position information and saves it in correspondence with each segmented word.

[0089] The server divides words with the same word position information into the same word position group, resulting in at least one word position group. Then, for each word position group, the server takes the first word in the word position group as the main word, and takes the other words in the word position group except the first word as synonyms of the main word.

[0090] In practical word segmentation, search engines can provide multiple different segmentation methods for the same segment of characters in the original search term. These different segmentation methods produce words with the same word position information, but different word content. For example, for a continuous Chinese string in the original search term, the search engine can segment it into both longer compound words and shorter sub-words contained within those compound words. These word segments of different lengths point to the same character range in the original search term, and therefore their word position information is the same. In this case, these different word segments pointing to the same character range are different members under the same word position group.

[0091] For example, assuming the original search term is "semiconductor," the word segmentation results and word position information returned by the search engine's word segmentation unit include: word position 0-2 for "semiconductor," word position 0-0 for "half," and word position 1-2 for "conductor." Here, "half" and "conductor" are sub-words obtained by the search engine from different granularities of character segmentation within the range of "semiconductor," and all three point to the same character range in the original search term (each consisting of three characters of that word). The word position information of these three segments is not entirely identical: word position 0-2 for "semiconductor," word position 0-0 for "half," and word position 1-2 for "conductor." Therefore, they cannot be simply grouped based on identical word positions. The server needs to determine the main word based on its position at the beginning of the word position group. "At the beginning" refers to the word that appears first in the list returned by the original word segmentation results, not the starting position of the word position range.

[0092] In one embodiment, the server obtains either the first link processing result or the second word segmentation link processing result. The first link processing result or the second word segmentation link processing result contains at least one word segmentation fragment, and the corresponding word position information for each word segmentation fragment. The word position information is used to indicate the start and end positions of the corresponding word segmentation fragment in the original search term, and is represented in the form of character offsets. Taking the Chinese search term "semiconductor equipment" as an example, the search engine word segmentation unit may return the following word segmentation results and corresponding word position information: "semiconductor" corresponds to word position (0,2), "equipment" corresponds to word position (3,4), "semiconductor equipment" corresponds to word position (0,4), "half" corresponds to word position (0,0), "conductor" corresponds to word position (1,2), "reserve" corresponds to word position (4,4), etc.

[0093] After obtaining the word segmentation results, the server first groups the segmented fragments based on their starting positions. Specifically, the server groups all segmented fragments with the same starting position into the same segmented group. For example, segmented fragments starting at position 0 include "semiconductor" (0,2), "semiconductor equipment" (0,4), and "half" (0,0), and these three fragments are grouped into the same segmented group; segmented fragments starting at position 3 include "equipment" (3,4), forming a separate group; and segmented fragments starting at position 4 include "preparation" (4,4), forming a separate group. In this way, each segmented group corresponds to the starting position of a character in the original search term, and multiple segmented fragments within the same group are different results produced by segmenting the starting position and subsequent characters at different granularities.

[0094] For each word segment group, the server, according to the order in the search engine's word segmentation unit results (i.e., the order after the search engine's internal scoring or weighting), designates the first word segment in the group as the main word, and the other word segments following the first as candidate words (or appositives) for the main word. These candidate words and the main word point to the same starting position in the original search term, and are products of different granularities of segmentation of the same character range. For example, in the word segment group starting at position 0, if the search engine returns the order "semiconductor equipment" (complete phrase), "semiconductor" (intermediate granularity), and "half" (single character granularity), then the server will determine "semiconductor equipment" as the main word, and "semiconductor" and "half" as candidate words for the main word.

[0095] It should be noted that the "candidate words" or "synonyms" mentioned here are different from "synonyms" in the sense of natural language processing. Candidate words only represent different segments generated by the search engine from the same character range in the original search terms with different granularities of segmentation, and they are not necessarily semantically equivalent to the main word. For example, when "semiconductor equipment" is segmented into "semi" and "conductor," "semi" and "semiconductor equipment" are not semantically equivalent and cannot be directly used as synonyms. Therefore, after the server completes the word grouping and determines the main word and candidate words, it still needs to further screen and process the candidate words to distinguish between candidate words that can be used as synonyms and noise segments that need to be discarded.

[0096] Specifically, after completing the word grouping, the server performs secondary splitting processing on the candidate words in each word group according to the character granularity. The secondary splitting processing determines what subsequent operation to take for each candidate word based on the character length of the candidate word and the morphological characteristics of the original search term itself.

[0097] For candidate words with a character length greater than or equal to 2 (e.g., "semiconductor") in the word group, it is highly reasonable to use them as synonyms of the main word because these candidate words have a certain degree of semantic integrity. The server retains these candidate words and proceeds to the synonym matching process. In the synonym matching process, the server matches the candidate word with a preset synonym dictionary to obtain the corresponding domain synonyms, thus expanding the scope of the search recall.

[0098] For single-character candidate words with a length of 1 character in the word group (e.g., "semiconductor", "conductor", "body"), the server determines the processing strategy based on the morphological characteristics of the original search term. If the original search term itself does not contain spaces (i.e., the original search term is a continuous string without spaces, such as "semiconductor equipment"), it indicates that the single-character candidate word is a noise fragment generated by the search engine over-segmenting complete semantic units. If it is directly used as a synonym in the search, it will lead to the recall of a large number of irrelevant documents, seriously impairing the search accuracy. Therefore, the server discards the single-character candidate word and does not include it in the search query conditions. At the same time, the server triggers a synonym completion process based on the main word corresponding to the single-character candidate word (e.g., "semiconductor equipment"), obtains a complete list of synonyms for the main word (e.g., "chip equipment", "integrated circuit equipment") from a preset thesaurus or knowledge graph, and adds the completed synonyms to the search query conditions to compensate for the recall coverage that may be lost due to discarding the single-character candidate word.

[0099] If the original search term itself contains spaces (e.g., "semiconductor"), it means that the user intends to input each single character as an independent search unit. In this case, the server retains the single-character candidate word and uses it as an independent main word in subsequent searches. At the same time, the server performs key word marking and synonym matching processes on the single-character candidate word.

[0100] After the aforementioned secondary splitting process, the server marks the main words and candidate words retained in each word group as key words. Specifically, the server matches the main words in each word group against a pre-defined key word lexicon. If the main word matches the key word lexicon, the server adds a key word tag to the main word, classifying it as a key word type. For retained multi-character candidate words, the server also matches them against the key word lexicon and adds a key word tag if a match is successful. For single-character candidate words, if the original input does not contain spaces, the server discards them directly without marking them as key words; if the original input contains spaces, the server matches the single-character candidate word against the key word lexicon to determine whether to mark it.

[0101] For example, assume that the original search term is "semiconductor equipment", and the word segmentation results returned by the search engine word segmentation unit in the arrangement order are: ["semiconductor equipment" (0,4), "semiconductor" (0,2), "semi" (0,0), "equipment" (3,4), "equipment" (4,4), "conductor" (1,2)]. After the server groups the results by the starting position, the group with the starting position 0 includes "semiconductor equipment", "semiconductor", and "semi", where "semiconductor equipment" is taken as the main term, and "semiconductor" and "semi" are taken as candidate terms; the group with the starting position 3 includes "equipment", which is taken as the main term, and "equipment" is taken as a candidate term; the group with the starting position 1 includes "conductor", but there is only one word segmentation fragment in this group, so "conductor" is directly taken as the main term. Then secondary shunting is performed: the candidate term "semiconductor" (length > 1) in the group with the starting position 0 is retained and enters synonym matching; the candidate term "semi" (length = 1) is discarded because the original input contains no spaces, and synonym completion based on the main term "semiconductor equipment" is triggered. The candidate term "equipment" (length = 1) in the group with the starting position 3 is also discarded because the original input contains no spaces, and synonym completion based on the main term "equipment" is triggered. Finally, key term marking is performed on the retained main terms and candidate terms to obtain a structured term set.

[0102] No matter what specific grouping method is adopted, the core idea is consistent: the flat word segmentation list output by the second word segmentation link, which may have overlaps and redundancy, is converted into a hierarchical structural relationship of main terms and synonyms through the association of lexical position information, so that each semantic unit corresponds to one main term and at least one synonym.

[0103] For example, assume that the word segmentation results returned by the search engine word segmentation unit for "domestic substitution" are ["domestic substitution" (0-3), "domestic" (0-1), "substitution" (2-3)], the server groups these word segments by the starting position and lexical position association, takes "domestic substitution" as the main term, and takes "domestic" and "substitution" as synonyms of this main term. Through this lexical position grouping process, the word segmentation results of the search engine are no longer a flat string array, but form a structured term set with semantic hierarchical relationships.

[0104] In another example, assuming the original search term is "domestic substitution of semiconductor equipment", the word segmentation results and corresponding word position information returned by the search engine's word segmentation unit are: ["semiconductor" (0-2), "equipment" (3-4), "domestic" (5-6), "substitution" (7-8), "semiconductor equipment" (0-4), "domestic substitution" (5-8)]. After the server groups the words according to their starting positions and word position intervals, it groups "semiconductor equipment" (0-4) and "semiconductor" (0-2) into the same word position group (both starting at position 0). The word at the beginning of this group is "semiconductor equipment", so "semiconductor equipment" is used as the main word and "semiconductor" as a synonym. Similarly, it groups "domestic substitution" (5-8) and "domestic" (5-6) into the same word position group (both starting at position 5), with "domestic substitution" as the main word and "domestic" and "substitution" as synonyms. The resulting hierarchical term set includes the two main words "semiconductor equipment" and "domestic substitution", as well as their respective synonyms.

[0105] After determining the main words based on word grouping, the server further determines the key word tags corresponding to each main word based on the main words in each word group, and attaches the key word tags to the corresponding main words. Key word tags are used to determine the segmentation type of each word (i.e., the main word), that is, to distinguish between key word types and non-key word types.

[0106] The beneficial effect of this embodiment is that, since the key word tag is directly attached to the main word, when determining the segmentation type of each word, it is only necessary to check whether the segment carries the key word tag to quickly complete the determination, thus improving processing efficiency. Through this word position grouping and main word synonym mapping mechanism, even if the AI ​​word segmentation link is unavailable due to timeout or other reasons, the results of the search engine word segmentation link can still support the construction of subsequent query conditions in a structured manner, ensuring the stability of the retrieval system and the quality of results.

[0107] In one optional embodiment, determining the segmentation type of each segment in the first link processing result or the second segmentation link processing result includes: for each segment in the first link processing result or the second segmentation link processing result, if the segment carries a key word tag, determining the segmentation type of the segment is a key word type; if the segment does not carry a key word tag, determining the segmentation type of the segment is a non-key word type.

[0108] The server determines the segmentation type for each segment in the first or second segmentation processing result. Segmentation types include two categories: key word type and non-key word type. The determination is based on whether the segment carries a key word tag. In some embodiments, the server iterates through each segment in the first or second segmentation processing result, checking each segment for a key word tag. The key word tag, as an attribute field of the segmentation, can be appended to the segmentation data structure as a Boolean flag or exist as an enumerated status field. The server completes the determination by reading the value of this attribute field.

[0109] Specifically, for any word segment in the first or second segmentation processing result, the server first obtains the key word tag field corresponding to that word. If the key word tag field is in a valid state (e.g., a flag of 1 or true), it indicates that the word segment has been marked as a key word, and the server determines that the word segmentation type is a key word type. If the key word tag field is in an invalid state (e.g., a flag of 0 or false), or if the word segment does not contain a key word tag field, it indicates that the word segment has not been marked as a key word, and the server determines that the word segmentation type is a non-key word type.

[0110] Understandably, the addition of key word tags is a product of the aforementioned word grouping and key word lexicon matching steps. When the server matches the business key word lexicon with the main word in each word group, it adds a key word tag to the data structure of that main word. Therefore, word segmentation type determination essentially transforms the previous key word tag addition result into a classification result, enabling subsequent matching weight allocation to be quickly decided based on the word segmentation type.

[0111] For example, suppose the first link processing fails to produce a result, while the second word segmentation link processing result contains three main words: "semiconductor equipment," "domestic substitution," and "industrial chain." "Semiconductor equipment" was previously tagged with a key word (is_keyword = true) because it matched the business key word lexicon. "Domestic substitution" and "industrial chain" did not match the key word lexicon and therefore did not carry key word tags. When determining the word segmentation type, the server detects that "semiconductor equipment" carries a key word tag, thus classifying it as a key word type; it detects that "domestic substitution" does not carry a key word tag, classifying it as a non-key word type; similarly, "industrial chain" is also classified as a non-key word type.

[0112] The beneficial effect of this embodiment is that word segments carrying key word tags are directly identified as key word types, while word segments without key word tags are identified as non-key word types. The determination logic is simple and clear, requiring no additional complex calculations or external queries, thus improving the server's processing efficiency. Since key word tags are attributes attached to word segments during word position grouping and key word lexicon matching, the word segmentation type determination in this embodiment only needs to read this attribute, avoiding repeated key word queries and matching operations, and reducing the system's computational overhead.

[0113] In one optional embodiment, after constructing the search query conditions based on the first or second word segmentation link processing results, the method further includes: adding the synonyms of each main word in the first or second word segmentation link processing results as extended matching items to the search query conditions.

[0114] The server adds the synonyms of each main word in the first or second word segmentation processing result as extended matching terms to the search query. In some embodiments, the word segmentation grouping processing has already established a corresponding synonym list for each main word. The sources of synonyms may include other word segments within the same word segmentation group in the search engine's word segmentation chain, excluding the main word, or terms from a business thesaurus, knowledge graph, vector nearest neighbor candidate words, or a manually maintained domain thesaurus mapping table. When constructing the search query, the server not only uses each main word itself as a matching term, but also includes all synonyms from the thesaurus list corresponding to each main word into the matching term set.

[0115] Specifically, for each main word in the first or second word segmentation processing result, the server obtains a list of synonyms corresponding to that main word. This list of synonyms can be generated during the word segmentation grouping step. When the server groups words with the same word segmentation information into the same word segmentation group, the first word in that group is considered the main word, and the remaining words are considered synonyms. Therefore, after the word segmentation step is completed, each main word is naturally associated with one or more synonyms. When constructing search query conditions, the server traverses all main words and groups each main word and its synonyms into a matching unit. Within the same matching unit, the main word and synonyms are on equal matching status. As long as a document matches any term in that unit, it is considered to have generated a valid match on that word segmentation group.

[0116] In actual search query construction, the server can use "OR" logic to place the main term and its synonyms in the same matching condition, allowing the main term and synonyms to share the same matching fields and the same matching weights. That is, when the server configures a main term with a matching weight of 1000 in the title field, all synonyms of that main term will also have a matching weight of 1000 in the title field. This configuration ensures that synonym hits receive the exact same score contribution as main term hits, thus achieving equivalent substitution of synonyms at the recall level.

[0117] For example, assuming the original search term is "domestic substitution of semiconductor equipment," the main terms determined after term grouping include "semiconductor equipment" and "domestic substitution." Synonyms for "semiconductor equipment" include "semiconductor," "equipment," and "chip equipment," while synonyms for "domestic substitution" include "domestic," "substitution," and "domestic-based substitution." When constructing the search query, the server uses "semiconductor equipment," "semiconductor," "equipment," and "chip equipment" as one matching unit, and "domestic substitution," "domestic," "substitution," and "domestic-based substitution" as another matching unit. When the server sets the matching weight of "semiconductor equipment" in the title field to 1000, "semiconductor," "equipment," and "chip equipment" also receive a matching weight of 1000 in the title field. Thus, a document titled "Analysis of the Chip Equipment Industry Chain," although not containing the main term "semiconductor equipment," can still receive a title matching contribution of 1000 points because it contains the synonym "chip equipment."

[0118] Based on the search query conditions, perform search matching in the target document library to obtain the search results corresponding to the original search terms, including: performing search matching in the target document library based on each main term and its synonym in the search query conditions; identifying documents in the target document library that match the main term or synonym as candidate documents, and obtaining the search results corresponding to the original search terms.

[0119] The server performs a search and matching process in the target document database based on the main terms and their synonyms in the search query. During the search and matching process, the search cluster scans each document in the target document database, checking whether each field (including title and body text fields) contains any of the terms from the search query. Here, "terms" include both the main terms themselves and the synonyms added as extended matching terms. Because the main terms and synonyms are configured to have the same matching fields and the same matching weights in the search query, the search cluster uses the exact same scoring method for main term hits and synonym hits when calculating document relevance scores.

[0120] Specifically, when traversing each document in the target document library, the retrieval cluster checks whether the document matches at least one term in each matching unit of the search query. If the document's title or body field contains the main term or any synonym of the matching unit, the retrieval cluster determines that the document matches that matching unit and adds a corresponding score to the document based on the field weight corresponding to that matching unit. If the document does not match any term in the matching unit, the retrieval cluster determines that the document does not match that matching unit and does not receive a score for that matching unit.

[0121] For example, continuing the previous example, suppose the search query includes matching unit A (main term "semiconductor equipment", synonyms "semiconductor", "equipment", "chip equipment") and matching unit B (main term "domestic substitution", synonyms "domestic", "substitution", "domesticization substitution"). The search cluster performs matching checks on each document in the target document library. Document X's title is "Research Report on Domestic Substitution of Semiconductor Equipment," which contains the main term "semiconductor equipment" from matching unit A and the synonym "domestic substitution" from matching unit B. Therefore, document X hits both matching units and obtains a high relevance score. Document Y's title is "Analysis of the Current Status of the Chip Equipment Industry Chain," which contains the synonym "chip equipment" from matching unit A, but does not contain any terms from matching unit B. Therefore, document Y only hits matching unit A and obtains the score corresponding to matching unit A. Document Z's title is "Accelerating the Process of Domestic Substitution," which contains the main term "domestic substitution" from matching unit B, but does not contain any terms from matching unit A. Therefore, document Z only hits matching unit B. Of the three documents, document X achieved the highest relevance score because it matched two units, while documents Y and Z achieved relatively lower scores because they matched only one unit. However, it's worth noting that without the synonym expansion mechanism, document Y would not have received a score for matching unit A at all because it does not contain "semiconductor device," resulting in a significantly lower relevance score and potentially rendering it unretrievable. Through synonym expansion, document Y was still effectively recalled because it contained the synonym "chip device."

[0122] After performing a search and matching operation, the server identifies documents in the target document library that match the main term or synonym as candidate documents, thus obtaining the search results corresponding to the original search terms. Specifically, after completing the matching detection of all documents in the target document library, the retrieval cluster filters out documents that match at least one term (main term or synonym) in at least one matching unit. These documents are identified as candidate documents and included in the subsequent sorting and output process. For documents that do not match any term in any matching unit, the retrieval cluster excludes them from the candidate document list.

[0123] The beneficial effect of this embodiment is that by adding the synonyms of each main word in the first or second word segmentation link processing result as extended matching items to the search query conditions, and performing search matching based on the main word and synonyms, documents that match the main word or synonyms are determined as candidate documents, thereby improving the retrieval recall rate.

[0124] In one optional embodiment, after obtaining the search results corresponding to the original search terms, the method further includes: determining the hit position of each word in the search query conditions in each candidate document; performing highlighting on each word in each candidate document according to the hit position of each word to obtain the highlighted segment corresponding to each candidate document; and returning each candidate document and its corresponding highlighted segment to the user.

[0125] The server determines the hit position of each word in the search query within each candidate document. This hit position refers to the specific character position occupied by the word (including the main word and its synonyms) in the text content of the candidate document, including the start and end positions.

[0126] In some embodiments, when performing a search match, the retrieval cluster not only returns the matched documents and their relevance scores, but also the specific location information of each matched term in each field of the document. Specifically, when the search analyzer of the retrieval cluster detects that a word in a document field successfully matches a term in the search query, it records the start and end offsets of that word in the text of that field and returns this location information along with the document to the server.

[0127] For example, suppose a candidate document is titled "Research Report on Domestic Substitution of Semiconductor Equipment," and the search query includes the terms "semiconductor equipment" and "domestic substitution." When performing a search match, the search cluster detects "semiconductor equipment" appearing in the 0th to 4th character positions of the title, and detects "domestic substitution" (i.e., a synonym of "domestic substitution") appearing in the 5th to 9th character positions of the title. The search cluster returns this hit information to the server, which then determines the hit position of each term in the candidate document.

[0128] Based on the corresponding hit positions of each word, the server performs highlighting on each word in each candidate document, resulting in highlighted segments for each candidate document. Highlighting refers to adding preset highlight style marks to the text at the hit positions within the text content of the candidate document, making these texts visually distinct from other text when displayed.

[0129] Specifically, for each field in each candidate document, the server locates the corresponding hit positions in the original text of that field based on the hit positions of each word segment in that field, and then inserts highlight tags before and after the hit text segments. For example, if the hit position of the word "semiconductor equipment" in the title field is from the 0th to the 4th character, the server inserts a highlight tag before the 0th character and after the 5th character in the title text, so that "semiconductor equipment" is highlighted in the title.

[0130] The beneficial effect of this embodiment is that by determining the corresponding hit position of each word in the search query conditions in each candidate document, precise location of the search hit is achieved. Since the word segmentation set on which the highlighting is based is exactly the same as the word segmentation set on which the search query conditions are constructed and the relevance score is calculated, the highlighted words that users finally see on the results page are consistent with the words that actually participate in sorting and filtering, thus improving the credibility of the search results display.

[0131] In an optional embodiment, the method further includes: if the input category to which the original search term belongs is a single keyword category, constructing search query conditions based on the original search term; and performing search matching in the target document library based on the search query conditions to obtain the search results corresponding to the original search term.

[0132] After performing character morphological feature analysis on the original search terms, if the input pre-classification module of the server determines that the input category to which the original search terms belong is a single keyword category, the server will not send the original search terms to the first or second word segmentation link for word segmentation processing. Instead, it will directly construct search query conditions based on the original search terms and then perform search matching in the target document library based on the search query conditions to obtain the search results corresponding to the original search terms.

[0133] In some embodiments, single-keyword category inputs include, but are not limited to: single Chinese short words (e.g., "semiconductor"), single consecutive English words (e.g., "artificial"), stock codes, phrases enclosed in quotation marks, number strings, and other single semantic units that do not contain spaces and do not require semantic splitting or segmentation. For this type of input, it inherently represents the user's complete query intent. Sending it into a word segmentation process would actually destroy its semantic integrity, leading to divergent recall results and decreased precise matching ability.

[0134] Understandably, the essential difference between single-keyword and multi-keyword categories lies in the fact that a single-keyword input is already a complete semantic unit, requiring no further identification of its multiple semantic segments; while a multi-keyword input contains two or more semantic units, necessitating semantic boundary identification and term splitting through word segmentation. Therefore, the most suitable approach for single-keyword input is to maintain its integrity, allowing it to participate in the retrieval directly in its original form. After marking it as a key word and matching synonyms, the aforementioned query construction, sorting, and highlighting processes corresponding to the multi-keyword category can be reused.

[0135] The beneficial effect of this embodiment is that, when the input category to which the original search term belongs is a single keyword category, all word segmentation links are skipped, and the search query conditions are directly constructed and search matching is performed based on the original search term, so that the single keyword sample avoids the loss of recall accuracy caused by excessive segmentation by the word segmenter.

[0136] In one optional embodiment, the original search terms are classified according to their character morphological features to obtain the input category to which the original search terms belong, including: detecting whether the original search terms contain spaces or separators; if the original search terms contain spaces or separators, determining that the input category to which the original search terms belong is a multi-keyword category; if the original search terms do not contain spaces or separators, detecting whether the original search terms contain quotation marks; and if the original search terms are contained in pairs of quotation marks, determining that the input category to which the original search terms belong is a single-keyword category.

[0137] After receiving the original search terms, the server's input pre-classification module first performs normalization preprocessing on the original search terms, and then classifies the original search terms according to their character morphological features to obtain the input category to which the original search terms belong. Specifically, the classification process is executed according to the following decision logic.

[0138] In one embodiment, the server first detects whether the original search term contains spaces or separators. A space refers to the English space character (U+0020) or the Chinese full-width space (U+3000); separators refer to punctuation marks used to separate different semantic units, such as commas (,), semicolons (;), pause marks (、), vertical bars (|), and forward slashes ( / ). If the server detects that the original search term contains any of the above spaces or separators, it determines that the original search term is composed of multiple independent semantic segments, and accordingly determines that the input category to which the original search term belongs is a multi-keyword category.

[0139] For example, suppose a user inputs the search term "domestic substitution of semiconductor equipment." The server detects that the input contains English spaces, recognizing "semiconductor equipment" and "domestic substitution" as two independent semantic segments, thus determining the input to belong to a multi-keyword category. Similarly, if the user inputs "artificial intelligence, machine learning, deep learning," the server detects that the input contains comma separators, determining that the input is a multi-keyword category composed of three keywords. Likewise, if the user inputs "photovoltaics|wind power|energy storage," the server detects vertical bar separators, similarly determining it to be a multi-keyword category.

[0140] Understandably, spaces and separators are the most intuitive markers for users to manually segment multiple keywords. When user input contains these markers, it's reasonable to infer that the user intends to retrieve different semantic units as independent matching dimensions, rather than treating them as a complete phrase. Therefore, classifying it as a multi-keyword category and incorporating it into the word segmentation process for semantic splitting and separate matching aligns with the user's intent.

[0141] If the server detects that the original search term does not contain spaces or separators, it further checks whether the original search term is enclosed in paired quotation marks. Quotation marks can be English quotation marks (") or Chinese quotation marks (“ ” or ' '). If the server detects quotation marks in the original search term, it indicates that the user wants to match the search term as a complete, exact phrase, rather than splitting it into multiple independent terms. Based on this, the server determines that the input category to which the original search term belongs is the single keyword category.

[0142] For example, suppose a user inputs the search term "artificial intelligence". The server detects that the input contains Chinese quotation marks and determines that the input category is a single keyword category. Under this category, the server will construct the search query condition using "artificial intelligence" as a complete phrase, requiring that the document must contain the complete string "artificial intelligence" continuously, rather than matching "artificial" and "intelligent" separately. This processing method ensures the accuracy of precise phrase queries and avoids false recalls caused by over-segmenting complete phrases. Similarly, if the user inputs "machine learning", the server detects English quotation marks and also determines it as a single keyword category, matching "machine learning" as a complete precise phrase.

[0143] In one embodiment, the server performs further judgment on single keyword inputs enclosed in paired quotation marks: if the original search term contains quotation marks and the quotation marks are empty or contain only whitespace characters, the server determines that the input is invalid, does not send it to the subsequent search process, and instead returns a prompt message to the user. This processing method avoids wasting server resources on invalid queries.

[0144] It should be noted that when the original search term contains neither spaces or separators nor quotation marks, the server needs to further determine whether the original search term belongs to a single keyword category or a multi-keyword category. Specifically, the server checks whether the original search term is a single Chinese short word, a single consecutive English word, a stock code, or a string of numbers; if so, the server determines that the input category to which the original search term belongs is the single keyword category. If the server detects that the original search term is a Chinese string without spaces but of relatively long length, containing multiple potential semantic segments (e.g., "domestic substitution of semiconductor equipment"), the server determines that the input category to which the original search term belongs is the multi-keyword category. This determination method, based on the above two basic detections, constitutes a complete input pre-classification logic, covering the accurate classification of various types of inputs without spaces or quotation marks.

[0145] The beneficial effect of this embodiment is that by detecting whether the original search terms contain spaces or separators, and further detecting whether they contain quotation marks when the original search terms do not contain spaces or separators, it can efficiently identify multi-keyword inputs and trigger the corresponding word segmentation processing links, avoiding over-segmentation and accuracy loss caused by sending quoted phrases into the word segmentation link. Furthermore, the determination is simple, involving only string matching and regular expression detection, resulting in low computational overhead.

[0146] Figure 4 For an overall interaction sequence diagram of a text retrieval method based on multi-link word segmentation in one embodiment, please refer to [link to diagram]. Figure 4 In a detailed embodiment, the interaction process involves the collaborative efforts of multiple functional entities, including the user / caller, the retrieval service, the word segmentation service, the Redis cache, and Elasticsearch. In the initial stage of the interaction process, the user or caller submits the original search term 'q' to the retrieval service. Upon receiving the search term, the retrieval service first sends a keyword parsing request to the word segmentation service. Upon receiving the request, the word segmentation service first performs a cache check, querying the Redis cache to see if a corresponding word segmentation cache result already exists for the original search term. Here, there is a first conditional branch: if the cache is hit, the Redis cache directly returns the pre-stored standard term result to the word segmentation service, which then skips the subsequent word segmentation calculation steps and directly returns the result to the retrieval service; if the cache is not hit, the word segmentation service further calls the Elasticsearch analyze interface to obtain the basic word segmentation result for the search term, using this as input for subsequent advanced processing.

[0147] After completing the basic word segmentation, the word segmentation service determines the input category based on the character morphological features of the original search term and triggers a second conditional branch. If the input category is determined to be a single keyword or a precise phrase, the word segmentation service does not need to perform complex semantic splitting and directly returns the original search term as a matching term to the retrieval service. If it is determined to be a multi-keyword category, the word segmentation service first calls the AI ​​word segmentation link to perform deep semantic word segmentation processing on the search term (see [link to relevant documentation]). Figure 4 The section marked with a monitoring circular arrow indicating "Calling AI Word Segmentation and Monitoring Timeout / Validity" aims to obtain more accurate semantic segmentation results. Here, the system incorporates a fault-tolerant circuit breaker mechanism, leading to a third conditional branch: If AI word segmentation is successful, the segmentation service will extract the main word, synonyms, and key word tags from the AI ​​segmentation results in a structured manner and return them to the retrieval service; however, in the event of AI timeout or invalidation, to ensure the overall availability of the system, the segmentation service will automatically trigger the circuit breaker fallback mechanism, stop waiting for AI word segmentation results, and immediately downgrade to a stable Elasticsearch word segmentation pipeline (i.e.,...). Figure 4 The code "reverts to the ES word segmentation link" and calls Elasticsearch to obtain the word position and appositive results corresponding to the search term, thus completing the downgraded word segmentation process.

[0148] After the complete processing chain described above, whether it's a direct return from a cache hit, a direct return from a single keyword match, a return from successful AI word segmentation, or a return from a fallback to the ES word segmentation chain, all paths ultimately converge on the step of "returning standard term results." The word segmentation service outputs standard structured terms that can be used to construct search query conditions to the search service. After obtaining the standardized term results, the search service uses this structured information and pre-configured weights to construct the corresponding search DSL query statement and sends a search execution request to Elasticsearch. Elasticsearch performs search matching in the target document library, calculates the relevance score of each document, and then returns the matched candidate documents and their relevance scores to the search service. Finally, the search service performs a composite sorting of the results based on the relevance score and time information, and simultaneously performs highlighting in the document content according to the matched position, returning the final search results after sorting and highlighting to the user or caller.

[0149] It should be emphasized that, first, after the word segmentation service obtains the word segmentation result (whether returned by the AI word segmentation link or the ES word segmentation link) and before entering the word position grouping and main word determination process, the server first performs a stop word filtering operation. The server loads a preset stop word list, which includes common meaningless function words such as "de", "le", "ma", "ne", "a", "o", and general words with no actual retrieval value such as "shi", "zai", "yu", "huo". For each word segmentation segment in the word segmentation result, the server matches it with the stop word list; if the word segmentation hits the stop word list, the server directly discards it, and it will not participate in subsequent word position grouping, main word determination, key word marking, and retrieval query condition construction. It should be noted that when a certain word segmentation segment exists in both the stop word list and the business key word lexicon, the server processes it according to the priority retention principle, that is, key word marking takes precedence over stop word filtering, and a word segmentation that has been marked as a key word is retained even if it hits the stop word list, so as to avoid the loss of core concepts caused by erroneous stop word filtering.

[0150] Second, in the key word marking stage, the server determines whether a word segmentation should be identified as a key word type by integrating two sources. The first source is the key word candidate flag output by the AI word segmentation link, that is, the candidate suggestion "whether this term should enter the key matching set" synchronously given by the AI semantic model during word segmentation; the second source is the exact matching result of the business key word lexicon, that is, whether the word segmentation hits the pre-set domain core term lexicon of the server. The server performs an "OR" (union) logical judgment on the above two sources: if a certain word segmentation carries the AI key word candidate flag, or the word segmentation hits the business key word lexicon, the server identifies the word segmentation as a key word type and attaches a key word tag to it. Through this union determination mechanism, even if a certain core domain term is not marked as a key candidate when output by the AI model (for example, due to insufficient coverage of model training data), the server can still supplementarily mark it as a key word through business lexicon matching; conversely, even if a certain word segmentation is not in the pre-set business lexicon, but the AI model judges that it has high importance according to the context semantics, the server also adopts the AI's suggestion to mark it as a key word, thereby realizing the mutual complementation of model intelligent judgment and business rule constraints.

[0151] Third, after completing all processing logic, the word segmentation service writes the final standard term results (including the main term list, the candidate / synonym list corresponding to each main term, key word tags, and the word segmentation type of each term) into the request context of the current retrieval request. This request context accompanies the entire lifecycle of the retrieval request and is shared by various business sub-libraries in the subsequent retrieval execution phase. When a retrieval request needs to perform retrieval matching in multiple business sub-libraries of the target document library (such as research report sub-libraries, news sub-libraries, announcement sub-libraries, knowledge base sub-libraries, etc.), the retrieval execution modules of each business sub-library read the standard term results from the same request context, rather than independently initiating duplicate word segmentation requests to the word segmentation service. On the one hand, this ensures that the same search term uses completely consistent word segmentation results and key word tags across different business sub-libraries, avoiding the problems of term differences and inconsistent search results across sub-libraries caused by multiple word segmentations; on the other hand, it reduces the overhead of repeated word segmentation calculations and the number of external service calls, improving the system throughput and response speed in multi-sub-library retrieval scenarios.

[0152] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0153] Based on the same inventive concept, this application also provides a method for implementing the above-mentioned text retrieval method based on multi-link word segmentation. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of a text retrieval device based on multi-link word segmentation provided below can be found in the limitations of the text retrieval method described above, and will not be repeated here.

[0154] In one exemplary embodiment, such as Figure 5 As shown, a text retrieval method apparatus based on multi-link word segmentation is provided, comprising:

[0155] The receiving module 501 is used to receive the original search terms and classify the original search terms according to the character morphological features of the original search terms to obtain the input category to which the original search terms belong.

[0156] The first word segmentation processing module 502 is used to segment the original search term using the first word segmentation link when the input category to which the original search term belongs is a multi-keyword category, and to obtain the first link processing result;

[0157] The second word segmentation processing module 503 is used to process the original search term using the second word segmentation link when the first link processing result indicates that the first word segmentation link has timed out, is abnormal, or is invalid, so as to obtain the second word segmentation link processing result; wherein, the word segmentation processing quality of the first word segmentation link is higher than that of the second word segmentation link.

[0158] Module 504 is used to construct search query conditions based on the results of the first or second word segmentation link processing.

[0159] The retrieval module 505 is used to perform retrieval matching in the target document library according to the retrieval query conditions and obtain the retrieval results corresponding to the original retrieval terms.

[0160] In an alternative embodiment, the construction module 504 is used for:

[0161] Obtain at least one field from the target document library that is involved in the search matching. The at least one field includes a first type of field and a second type of field. The first type of field contains the title information of the target document library, and the second type of field contains the body information of the target document library.

[0162] Determine the segmentation type of each word in the first or second segmentation process result; the segmentation type includes key word type and non-key word type.

[0163] Based on the word segmentation type, determine the matching weight of each word segment in each field; among them, for the same field, the matching weight of the word segment corresponding to the key word type is greater than the matching weight of the word segment corresponding to the non-key word type; for the same word segment, the matching weight of the word segment in the first type of field is greater than the matching weight of the word segment in the second type of field.

[0164] Based on each word segment and its matching weight in each field, search query conditions are generated.

[0165] In an optional embodiment, the retrieval module 505 is used to:

[0166] Based on the matching weight of each word in each field contained in the search query conditions, a search and matching is performed in the target document library to determine multiple candidate documents and their respective relevance scores.

[0167] Obtain the time information for each candidate document, where the time information represents the publication time or update time of the corresponding candidate document;

[0168] Based on the relevance scores of each candidate document, the candidate documents are arranged in descending order of relevance scores.

[0169] Candidate documents with the same relevance score are sorted according to the time information from newest to oldest to determine the sorting position of each candidate document;

[0170] Output each candidate document according to its respective sorting position to obtain the search results corresponding to the original search terms.

[0171] In an optional embodiment, it further includes:

[0172] The word position information acquisition module is used to acquire the word position information corresponding to each word in the first link processing result or the second word segmentation link processing result; the word position information is used to indicate the word position of the corresponding word in the original search term;

[0173] The segmentation module is used to group words with the same word position information into the same word position group, resulting in at least one word position group.

[0174] The word segmentation determination module is used to determine the word segment at the beginning of each word segment group as the main word, and the synonyms of the other words in the word segment group except the first word as the main word.

[0175] The tag determination module is used to determine the key word tags corresponding to each main word in each word group, and to attach the key word tags to the corresponding main words; among them, the key word tags are used to determine the segmentation type of each segmentation.

[0176] In an alternative embodiment, the construction module 504 is used for:

[0177] For each word segment in the first link processing result or the second word segmentation link processing result, if the word segment carries a key word tag, the word segmentation type of the word segment is determined to be a key word type;

[0178] If the word segment does not carry a key word tag, the word segmentation type is determined to be a non-key word type.

[0179] In an optional embodiment, it further includes:

[0180] The extension module is used to add the synonyms of each main word in the first or second word segmentation link processing results as extended matching items to the search query conditions;

[0181] The retrieval module 505 is used for:

[0182] Based on the main terms and their synonyms in the search query, perform a search and match in the target document database.

[0183] Documents in the target document library that match the subject or synonym are identified as candidate documents, and the search results corresponding to the original search terms are obtained.

[0184] In an optional embodiment, it further includes:

[0185] The hit location determination module is used to determine the hit location of each word in the search query conditions in each candidate document;

[0186] The highlighting module is used to highlight each word in each candidate document according to the hit position of each word, so as to obtain the highlighted segment corresponding to each candidate document.

[0187] The return module is used to return each candidate document and its corresponding highlighted segment to the user.

[0188] In an optional embodiment, it further includes:

[0189] The single keyword construction module is used to construct search query conditions based on the original search terms when the input category to which the original search term belongs is a single keyword category.

[0190] The single-keyword retrieval module is used to perform retrieval matching in the target document library based on the retrieval query conditions, and obtain the retrieval results corresponding to the original retrieval terms.

[0191] In an optional embodiment, the receiving module 501 is configured to:

[0192] Check if the original search terms contain spaces or separators;

[0193] If the original search terms contain spaces or separators, determine that the input category to which the original search terms belong is a multi-keyword category;

[0194] If the original search terms do not contain spaces or separators, check whether the original search terms contain quotation marks.

[0195] If the original search term is enclosed in paired quotation marks, the input category to which the original search term belongs is determined to be a single keyword category.

[0196] The modules in the aforementioned text retrieval device based on multi-link word segmentation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0197] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores target document libraries, term analysis results generated during term processing, key term libraries, and thesaurus data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a text retrieval method based on multi-link word segmentation.

[0198] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0199] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0200] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0201] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0202] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0203] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0204] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0205] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A text retrieval method based on multi-link word segmentation, characterized in that, The method includes: Receive the original search terms and classify them according to their character morphological features to obtain the input category to which the original search terms belong; When the input category to which the original search term belongs is a multi-keyword category, the original search term is segmented using the first word segmentation link to obtain the first link processing result; If the first segmentation link's processing result indicates that the first segmentation link's segmentation processing of the original search term timed out, was abnormal, or was invalid, then the second segmentation link is used to segment the original search term to obtain the second segmentation link's processing result; wherein, the segmentation processing quality of the first segmentation link is higher than that of the second segmentation link; Based on the first or second word segmentation link processing result, construct retrieval query conditions; Based on the search query conditions, a search and matching process is performed in the target document library to obtain the search results corresponding to the original search terms.

2. The method according to claim 1, characterized in that, Based on the first or second word segmentation link processing result, search query conditions are constructed, including: Obtain at least one field from the target document library that participates in the search matching. The at least one field includes a first type of field and a second type of field. The first type of field contains the title information of the target document library, and the second type of field contains the body information of the target document library. Determine the segmentation type of each word in the first link processing result or the second word segmentation link processing result; the segmentation type includes key word type and non-key word type; Based on the word segmentation type, the matching weight of each word segment in each field is determined; wherein, for the same field, the matching weight of the word segment corresponding to the key word type is greater than the matching weight of the word segment corresponding to the non-key word type; for the same word segment, the matching weight of the word segment in the first type of field is greater than the matching weight of the word segment in the second type of field; Based on each of the word segments and their matching weights in each of the fields, search query conditions are generated.

3. The method according to claim 2, characterized in that, Based on the search query conditions, a search and matching process is performed in the target document library to obtain the search results corresponding to the original search terms, including: Based on the matching weight of each word segment in each field contained in the search query conditions, a search matching is performed in the target document library to determine multiple candidate documents and their respective relevance scores. Obtain the time information of each candidate document, wherein the time information represents the publication time or update time of the corresponding candidate document; Based on the relevance score of each candidate document, the candidate documents are arranged in descending order of their relevance scores. Candidate documents with the same relevance score are arranged in order from newest to oldest according to the time information, and the sorting position of each candidate document is determined. Each candidate document is output according to its respective sorting position to obtain the search results corresponding to the original search terms.

4. The method according to claim 2, characterized in that, Also includes: Obtain the word position information corresponding to each word in the first link processing result or the second word segmentation link processing result; the word position information is used to indicate the word position of the corresponding word in the original search term; Word segments with the same word position information are grouped into the same word position group to obtain at least one word position group; For each word group, the first word in the word group is taken as the main word, and the other words in the word group except the first word are taken as synonyms of the main word. Based on the main word in each of the word groupings, determine the key word tag corresponding to each main word, and attach the key word tag to the corresponding main word; wherein, the key word tag is used to determine the segmentation type of each segmentation.

5. The method according to claim 4, characterized in that, Determine the segmentation type of each word in the first link processing result or the second word segmentation link processing result, including: For each word segment in the first link processing result or the second word segmentation link processing result, if the word segment carries a key word tag, the word segmentation type of the word segment is determined to be the key word type; If the word segment does not carry the key word tag, the word segmentation type is determined to be the non-key word type.

6. The method according to claim 4, characterized in that, After constructing the retrieval query conditions based on the first or second word segmentation link processing result, the method further includes: The synonyms of each main word in the first or second word segmentation link processing result are used as extended matching items and added to the search query conditions. Based on the search query conditions, a search and matching process is performed in the target document library to obtain the search results corresponding to the original search terms, including: Based on each of the main terms and their respective synonyms in the search query conditions, a search and matching process is performed in the target document library. Documents in the target document library that match the subject or synonym are identified as candidate documents, and the search results corresponding to the original search terms are obtained.

7. The method according to claim 3, characterized in that, After obtaining the search results corresponding to the original search terms, the process also includes: In each of the candidate documents, determine the hit position corresponding to each of the word segments in the search query conditions; Based on the hit position of each word segment, highlighting is performed on each word segment in each candidate document to obtain the highlighted segment corresponding to each candidate document; Each candidate document and its corresponding highlighted segment are returned to the user.

8. The method according to claim 1, characterized in that, Also includes: If the input category to which the original search term belongs is a single keyword category, then search query conditions are constructed based on the original search term. Based on the search query conditions, a search and matching process is performed in the target document library to obtain the search results corresponding to the original search terms.

9. The method according to any one of claims 1-7, characterized in that, The original search terms are classified according to their character morphological features to obtain the input category to which the original search terms belong, including: Detect whether the original search terms contain spaces or separators; If the original search term contains spaces or separators, the input category to which the original search term belongs is determined to be a multi-keyword category; If the original search term does not contain spaces or separators, detect whether the original search term contains quotation marks. If the original search term is enclosed in paired quotation marks, the input category to which the original search term belongs is determined to be a single keyword category.

10. A text retrieval device based on multi-link word segmentation, characterized in that, The device includes: The receiving module is used to receive the original search terms and classify the original search terms according to the character morphological features of the original search terms to obtain the input category to which the original search terms belong; The first word segmentation processing module is used to segment the original search term using a first word segmentation link when the input category to which the original search term belongs is a multi-keyword category, and to obtain the first link processing result; The second word segmentation processing module is used to perform word segmentation processing on the original search term using the second word segmentation link when the processing result of the first link indicates that the word segmentation processing of the first word segmentation link for the original search term has timed out, is abnormal, or is invalid, so as to obtain the processing result of the second word segmentation link; wherein, the word segmentation processing quality of the first word segmentation link is higher than that of the second word segmentation link. The module is used to construct search query conditions based on the results of the first or second word segmentation link processing. The retrieval module is used to perform retrieval matching in the target document library according to the retrieval query conditions, and obtain the retrieval results corresponding to the original retrieval terms.

Images