Method and device for determining section selection of highlight text and program product

Abstract

The invention discloses a highlight text section selection determination method and device and a program product, and the method comprises the steps: obtaining at least one keyword; obtaining at least one document fragment in the document to be highlighted; determining a highlight value of the obtained document fragment based on the distribution condition of the keyword in the document fragment; a highlighted text segment is determined based on the document segments whose highlighted values reach a threshold.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and program product for determining a highlighted text selection. Background Art

[0002] In the field of information retrieval, facing massive amounts of document data, how to efficiently and accurately find the information users need has always been a core and extremely challenging problem. With the rapid development of information technology, the number and types of documents have exploded, which has undoubtedly increased the difficulty of information retrieval. Traditional information retrieval methods mostly rely on precise matching of keywords, that is, the system will search for strings in the document that are exactly the same as the keywords entered by the user. However, this simple matching method often fails to find the text that best meets the user's intention, and the matching accuracy and efficiency are low. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide a method, device and program product for determining a highlighted text selection.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] On the one hand, an embodiment of the present application provides a method for determining a highlighted text selection, the method comprising: obtaining at least one keyword; obtaining at least one document fragment in the document to be highlighted; determining a highlight value of the obtained document fragment based on the distribution of the keywords in the document fragment; and determining a highlighted text selection based on the document fragment whose highlight value reaches a threshold.

[0006] On the other hand, an embodiment of the present application provides a method for determining a highlighted text selection, the method comprising: in response to receiving a highlighted text selection event for at least one keyword, displaying a highlighted text selection; wherein the highlighted text selection is determined based on the document fragment in the document to be highlighted whose highlight value reaches a threshold; and the highlight value is determined based on the distribution of the keyword in the document fragment.

[0007] On the other hand, an embodiment of the present application provides a device for determining a highlighted text selection, the device comprising: a first acquisition module, used to acquire at least one keyword; a second acquisition module, used to acquire at least one document fragment in the document to be highlighted; a first determination module, used to determine a highlight value of the acquired document fragment based on the distribution of the keyword in the document fragment; and a second determination module, used to determine a highlighted text selection based on the document fragment whose highlight value reaches a threshold.

[0008] On the other hand, an embodiment of the present application provides a device for determining a highlighted text selection, the device comprising: a display module, for displaying a highlighted text selection in response to receiving a highlighted text selection event for at least one keyword; wherein the highlighted text selection is determined based on the document fragment whose highlight value reaches a threshold in the document to be highlighted; and the highlight value is determined based on the distribution of the keyword in the document fragment.

[0009] On the other hand, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements some or all of the steps in the above method.

[0010] In the embodiment of the present application, by obtaining at least one document fragment in the document to be highlighted, different parts of the document can be covered, increasing the possibility of finding content related to the keyword; based on the distribution of the keyword in the document fragment (such as the frequency of occurrence, position, contextual relevance, etc.), the highlight value of the obtained document fragment is determined, and the matching degree and relevance of each document fragment with the keyword can be quantitatively evaluated, thereby screening out the fragments that are more likely to contain the information required by the user; finally, based on the document fragment whose highlight value reaches the threshold, the highlighted text selection is determined, which can improve the effect that the output text is in line with the user's query intention, and improve the accuracy and efficiency of the search. In this way, the present application realizes the rapid and accurate selection of the text content most relevant to the user's query from a large number of documents, and optimizes the search experience.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0013] Figure 1 A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 1 ;

[0014] Figure 2 A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 2 ;

[0015] Figure 3 A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 3 ;

[0016] Figure 4A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 4 ;

[0017] Figure 5 A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 5 ;

[0018] Figure 6 A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 6 ;

[0019] Figure 7 A schematic diagram of the implementation flow of another method for determining a highlighted text selection provided in an embodiment of the present application;

[0020] Figure 8 A schematic diagram of an implementation of a sliding window algorithm provided in an embodiment of the present application;

[0021] Fig. 9 An example diagram of a scene of an optimal selection provided in an embodiment of the present application;

[0022] Fig.10 A scene example diagram of a suboptimal segment provided in an embodiment of the present application;

[0023] Fig.11 An example diagram of a scene of splicing selected segments provided in an embodiment of the present application;

[0024] Fig.12 A schematic diagram of the structure of a device for determining a highlighted text selection provided in an embodiment of the present application;

[0025] Fig.13 A schematic diagram of the composition structure of another device for determining a highlighted text selection provided in an embodiment of the present application;

[0026] Fig.14 A hardware entity diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0028] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0030] The highlighting mechanism is a key step in the full-text search system. It involves selecting a context segment containing the user query from the original document that has been recalled, and visually distinguishing the user query part in this context by highlighting the color. This process directly affects the user's intuitive judgment of the accuracy of the search results. Although the accuracy of the search results has been preliminarily determined in the recall stage (that is, the system has screened out the documents containing the user query according to a certain algorithm), the highlighting process plays a vital role in improving the user experience in the subsequent process. The core challenge is how to optimize the highlighting display to ensure that as many keywords in the user query as possible are included in the highlighted segment, thereby enhancing the user's perception and making him feel that the search results are highly consistent with his query intent. If the highlighted segment fails to fully display all or key parts of the user's query (although the original recalled document does contain these query terms, some terms may not be selected for highlighting due to reasons such as algorithm selection or segment length limitation), the user may mistakenly believe that the search results are inaccurate. Therefore, optimizing the highlighting strategy to ensure the effective display of query terms is crucial to improving the overall performance and user satisfaction of the full-text search system.

[0031] For example, taking the user query "search classification" as an example, the keywords (tokens) obtained after word segmentation are "search" and "classification". The text matching mechanism of Elasticsearch (ES) recalls documents based on the co-occurrence of these two keywords (i.e., containing "search" AND "classification" at the same time). However, the specific location of these keywords in the document is uncertain, which brings challenges to highlighting. The current highlight selection logic tends to select paragraphs with more concentrated hit keywords to improve the calculation of text relevance scores. However, this method may cause some problems, such as "search" may appear multiple times in the highlighted paragraphs but lack "classification", or the edit distance between "search" and "classification" in the same paragraph (i.e., their actual interval in the text) is far. These situations are compared with the ideal scenario of "search classification" being completely and continuously hit, and the user experience will be greatly reduced. It is worth noting that "search classification" is only a simple and ideal query case. In actual applications, user queries may be more complex and diverse, including long queries, mixed numbers, English, punctuation marks, etc., and rarely directly correspond to complete sentences in the original text. As the query length increases, the number of tokens generated after word segmentation also increases, which further exacerbates the difficulty of highlighting. Although the current recall strategy ensures that all recalled documents contain the keywords in the user's query, the actual arrangement and spacing between tokens after word segmentation is uncertain. Especially when the highlight segment length is limited (such as 75 characters), it becomes particularly difficult to accurately and tightly highlight all the keywords in the user's query.

[0032] Therefore, in order to improve the user experience, the highlight strategy algorithm needs to be redesigned. The embodiment of the present application provides a method for determining a highlighted text selection, which can select a highlighted text selection based on the distribution of keywords in the document, rather than simply based on the number of times the keywords appear. In this way, the problem of inaccurate highlighted text selection can be more effectively solved, reducing the user's illusion of inaccurate recall caused by inaccurate highlighted text selection.

[0033] The embodiment of the present application provides a method for determining a highlighted text selection, which can be executed by a processor of a computer device. The computer device may refer to a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device), or other device with data processing capabilities.

[0034] Figure 1 A schematic diagram of an implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes the following steps S101 to S104:

[0035] Step S101: Obtain at least one keyword.

[0036] The keyword can be a word, phrase or string input by the user or selected by the system to describe information needs or content themes. In text processing and search engine technology, keywords are a direct reflection of user intent and the basis for system matching and searching.

[0037] In some embodiments, at least one keyword related to the user's intent can be obtained through user input, system recommendation, or predefined methods, so as to be matched and searched in the document later. In some implementation scenarios, the device can provide a user input interface to allow the user to enter one or more keywords. In other embodiments, the device can also intelligently recommend one or more keywords based on the user's search history, behavior pattern, or context information.

[0038] For example, suppose a user is using the search function of an online reading application and hopes to find books about "artificial intelligence". At this time, the user will enter "artificial intelligence" as a keyword in the search box, and the system will match and search in the database based on this keyword.

[0039] Step S102: Obtain at least one document segment in the document to be highlighted.

[0040] In some embodiments, the document to be highlighted is a document that needs to be searched and extracted, and is the original data from which the user hopes to find the highlighted text segment; wherein the document to be highlighted can be, but is not limited to, a table document (such as Excel, CSV, etc.), a text document (such as TXT, PDF, DOC, etc.) and a presentation document (such as PPT, etc.). The document segment is a text paragraph or sentence segmented from the document to be highlighted that contains specific information or meets specific conditions.

[0041] In some embodiments, keywords may be confirmed first, and then the document to be highlighted may be confirmed. That is, keywords that the user needs to focus on are obtained first, and then documents containing these keywords are searched and screened based on the keyword database, so as to obtain the document to be highlighted.

[0042] In other embodiments, the document to be highlighted is available first, and then the keywords are confirmed. That is, the existence of the document to be highlighted precedes the determination of the keywords. In an exemplary scenario, when the user already has or obtains a specific document collection, but has not yet determined which specific words to focus on, the keywords related to the content of the document to be highlighted can be extracted or selected based on the content, theme, or other needs of the user. The keywords can be, but are not limited to, subject words, high-frequency words, key phrases, etc. in the document to be highlighted.

[0043] In some embodiments, the document to be highlighted can be divided based on a fixed length to obtain the above-mentioned at least one document fragment. The implementation method of the above-mentioned step S102 can be: determine the fixed length of each document fragment; start from the starting position of the document to be highlighted, and divide it according to the fixed length until the end of the document; each divided continuous character sequence is a document fragment. Exemplarily, assuming that the total length of the document to be highlighted is 4000 characters, and the length of each document fragment is set to 100 characters, the document can be divided into 40 document fragments.

[0044] In other embodiments, the document to be highlighted can be intercepted based on a preset sliding window to obtain the above-mentioned at least one document fragment. The above-mentioned step S102 can be implemented as follows: determining the window length and sliding step of the sliding window; sliding the sliding window from the starting position of the document to be highlighted, intercepting a continuous character sequence in the current sliding window after each sliding to obtain a document fragment; moving backward by a unit of sliding step to obtain the next document fragment; until sliding to the end of the document. Exemplarily, assuming that the total length of the document to be highlighted is 4000 characters, the window length of the sliding window is set to 100 characters, and the sliding step is set to 1 character, the document can be divided into 3901 document fragments.

[0045] In some other embodiments, the document to be highlighted may be divided based on the symbols or formats in the document to be highlighted to obtain the above-mentioned at least one document fragment. The above-mentioned step S102 may be implemented as follows: identifying a specific symbol or format in the document to be highlighted; dividing the document to be highlighted into a plurality of continuous content blocks according to the position of the specific symbol or format in the document to be highlighted, and each content block is a document fragment. Exemplarily, for a document to be highlighted containing a plurality of chapters and paragraphs, it may be divided according to chapter titles and paragraph marks, and the content of each chapter or paragraph is a document fragment.

[0046] It should be noted that different document fragments may or may not include repeated document content, and the actual selection may be determined based on the search accuracy.

[0047] Step S103: Determine a highlight value of the acquired document fragment based on the distribution of the keywords in the document fragment.

[0048] In an embodiment of the present application, the distribution of the above-mentioned keywords in the document fragment includes but is not limited to: the number of occurrences of the keywords in the document fragment, the position of the occurrence of the keywords in the document fragment, the contextual association between the keywords and the document fragment, the proportion of the keywords in the document fragment, etc.

[0049] Among them, the more times a keyword appears in a document fragment, it usually means that the keyword is more closely related to the content of the document fragment; the position of the keyword in the document fragment refers to the specific position of the keyword in the document fragment, such as the beginning, middle or end. The keyword appearing in the beginning part can often better reflect the theme of the document fragment; the contextual association between the keyword and the document fragment actually reflects the association between the keyword and the surrounding words. If the degree of association between a keyword and the context is large, it indicates that the probability that the current document fragment is the expected query fragment is greater; the proportion of the keyword in the document fragment is the ratio of the total number of words appearing in the keyword in the document fragment to the total number of words in the document fragment. Generally speaking, this parameter is related to the length of the keyword. The more characters a keyword has, the greater the corresponding proportion of keywords with the same number of appearances.

[0050] In some embodiments, a machine learning model can be used to predict highlight values. First, the distribution characteristics of keywords are extracted from the document fragment, including the number of occurrences, location information, contextual associations, etc. Then, these features are used as input to train a machine learning model (such as logistic regression, support vector machine or neural network, etc.). During the training process, known relevant and non-relevant document fragments are used as training data, and the prediction accuracy of the highlight value is optimized by adjusting the model parameters. After the training is completed, the keyword distribution characteristics of the newly acquired document fragment are input into the model to obtain its highlight value.

[0051] In other embodiments, the scores corresponding to each of the above-mentioned distribution features (including the number of occurrences of the keyword in the document fragment, the position of the keyword in the document fragment, the contextual association between the keyword and the document fragment, and the proportion of the keyword in the document fragment) can also be calculated separately, and the highlight value of the document fragment can be determined by combining the scores corresponding to each distribution feature.

[0052] Step S104: Determine a highlighted text segment based on the document segment whose highlight value reaches a threshold.

[0053] In some embodiments, whenever a new document fragment is successfully obtained from the document to be highlighted, its highlight value is immediately calculated according to the distribution of keywords in the document fragment, and the document fragments whose highlight values ​​reach the threshold are retained, and the document fragments whose highlight values ​​do not reach the threshold are discarded. The above process is repeated until all document fragments are obtained and compared, thereby obtaining all the document fragments whose highlight values ​​reach the threshold. The threshold can be pre-configured, and the user can adjust the fuzziness of the search process by adjusting the size of the threshold.

[0054] In other embodiments, all document fragments may be first obtained from the document to be highlighted. Then, the highlight values ​​of these document fragments are uniformly calculated. A threshold is generated based on the highlight values ​​of all document fragments, and then the document fragments whose highlight values ​​reach the threshold are retained. Here, the threshold is dynamically changed, that is, it is automatically adjusted according to the overall distribution characteristics of the keyword highlight values ​​in the document fragment set. In this way, it can flexibly adapt to different document contents, thereby improving the accuracy and relevance of search results.

[0055] In some embodiments, after determining a document fragment whose highlight value reaches a threshold, the document fragment can be directly used as a highlighted text selection; in other embodiments, when there are multiple document fragments whose highlight values ​​are very close and all reach the threshold, these document fragments can be spliced ​​into one text fragment as a highlighted text selection.

[0056] In the embodiment of the present application, by obtaining at least one document fragment in the document to be highlighted, different parts of the document can be covered, increasing the possibility of finding content related to the keyword; based on the distribution of the keyword in the document fragment (such as the frequency of occurrence, position, contextual relevance, etc.), the highlight value of the obtained document fragment is determined, and the matching degree and relevance of each document fragment with the keyword can be quantitatively evaluated, thereby screening out the fragments that are more likely to contain the information required by the user; finally, based on the document fragment whose highlight value reaches the threshold, the highlighted text selection is determined, which can improve the output text to be more in line with the user's query intent, and improve the accuracy and efficiency of the search. In this way, the present application realizes the fast and accurate search of the text content most relevant to the user's query from a large number of documents, and optimizes the search experience.

[0057] In some embodiments, the document fragment whose highlight value reaches a threshold includes a fragment to be spliced ​​of each of the keywords; determining the highlighted text selection based on the document fragment whose highlight value reaches the threshold includes: in the case where there is no document fragment including at least two of the keywords in the document to be highlighted, splicing the fragments to be spliced ​​of each of the keywords to obtain the highlighted text selection.

[0058] The above embodiment is implemented when there is no document fragment directly including at least two of the keywords in the document to be highlighted, or in other words, the above embodiment is implemented when all the obtained document fragments include at most one keyword; by finding the fragments to be spliced ​​that are highly relevant to each keyword respectively and performing splicing processing, a text can be constructed that is not a continuous string existing in the document to be highlighted, but is highly matched with the user's query intention as a whole, thereby improving the perceived accuracy of information retrieval.

[0059] In the above embodiment, for each keyword, a segment to be spliced ​​can be determined from the obtained multiple document segments; wherein the segment to be spliced ​​for each keyword is a document segment that includes the keyword and has a highlight value that reaches a threshold and is the highest.

[0060] In some embodiments, the target length corresponding to each keyword can be determined based on the text length of the highlighted text selection and the number of keywords; then, using the target length corresponding to each keyword, characters of the target length are intercepted from the to-be-joined segment corresponding to each keyword to obtain the to-be-joined text corresponding to each keyword; the to-be-joined text corresponding to each keyword is spliced ​​to obtain the highlighted text selection.

[0061] For example, it is assumed that the keywords are "artificial intelligence", "machine learning" and "deep learning", and the segment to be spliced ​​corresponding to "artificial intelligence" is: "artificial intelligence is a branch of computer science that aims to develop computers that can simulate human intelligence..."; the segment to be spliced ​​corresponding to "machine learning" is: "machine learning focuses on allowing computer systems to learn and improve through data and statistical methods..."; the segment to be spliced ​​corresponding to "deep learning" is: "deep learning often achieves..." by using multi-layer neural networks to process and analyze data. If the text length of the highlighted text segment is 45, then when the number of keywords is 3, the target length of each keyword is 15, and then the keyword can be used as the center in the segment to be spliced ​​corresponding to the keyword, and a total of 15 characters of text can be intercepted forward or backward (or a combination of both), and the highlighted text segment obtained after splicing is: "artificial intelligence is a branch of computer science...machine learning focuses on allowing computer systems to...deep learning uses multi-layer neural networks to...". It can be seen that 15 characters are intercepted for each keyword to complete the splicing of the highlighted text segment.

[0062] In the embodiment of the present application, when there is no document segment directly containing at least two keywords in the document to be highlighted, segments to be spliced ​​that are highly relevant to each keyword can be found respectively. At the same time, by splicing these segments to be spliced, a text that is not a continuous string but highly matches the user's query intent as a whole can be constructed, thus significantly improving the accuracy of information retrieval.

[0063] Figure 2 This is a schematic diagram of the implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application. Figure 2 , the method can be executed by a processor of a computer device. Figure 1 , Figure 1 Step S102 in can be updated to step S201 to step S202, combining Figure 2 The steps shown are explained.

[0064] Step S201: Use a sliding window to slide from a starting position in the document to be highlighted.

[0065] The starting position is the first position where the sliding window starts to slide in the document to be highlighted. In some embodiments, the starting position may be the beginning of the document to be highlighted, that is, the position of the first character or the first word. In other embodiments, depending on the specific application scenario and requirements, the starting position may also be set to other positions in the document to be highlighted, such as skipping the title, abstract or introduction of the document and sliding the window directly from the main text.

[0066] In some embodiments, a sliding window of a fixed size may be set, starting from the beginning of the document, and moving a preset step size (which may be the number of characters, words, or sentences, etc.) each time until the entire document to be highlighted is traversed. The size of the sliding window may be the same as the length of the highlighted text segment.

[0067] For example, assuming that the document to be highlighted is "This is a test document...", the sliding window size is 4 characters, and the step length is 1 character. Then the sliding window will sequentially capture document fragments such as "This is a", "This is a test", and "A test".

[0068] Step S202: During the sliding process, the document segment is captured in the document to be highlighted based on the sliding window.

[0069] In some embodiments, after each movement of the sliding window, a character string of corresponding length is intercepted from the document as a document fragment according to the starting and ending positions of the sliding window. The starting and ending positions define the character string covered by the sliding window in the document to be highlighted, and the above interception process is actually to copy the character string covered by the sliding window from the document fragment.

[0070] Exemplarily, based on the above example, when the sliding window moves to "a test", the character string is intercepted as the current document fragment.

[0071] In an embodiment of the present application, by adopting the sliding window technology to slide from the starting position of the document to be highlighted, the entire document can be traversed continuously to ensure that all possible document fragments are covered without omission, thereby improving the completeness of document fragment acquisition; at the same time, during the sliding process, based on the position and size of the sliding window, the document fragment is intercepted in real time in the document to be highlighted, and multiple candidate document fragments can be efficiently generated, providing rich samples for the subsequent keyword-based highlight value calculation.

[0072] In some embodiments, the above step S103 can be updated to step S203.

[0073] Step S203: In response to intercepting the document fragment, determining a highlight value of the acquired document fragment based on the distribution of the keywords in the document fragment.

[0074] In the embodiment of the present application, each time a document segment is extracted from the document to be highlighted, its highlight value is immediately calculated based on the distribution of keywords in the document segment.

[0075] It can be understood that the relevant implementation of calculating the highlight value of the document fragment can refer to the relevant description in step S103.

[0076] In the embodiment of the present application, by calculating the highlight value of a document fragment immediately after it is intercepted, each document fragment can be evaluated in time after it is generated; when processing a large number of documents, if all possible document fragments are generated first and then the highlight values ​​are calculated uniformly, a large amount of memory and computing resources may be consumed. However, by adopting the calculation timing of "responding to intercepting the document fragment", each fragment can be processed step by step, avoiding the resource pressure caused by processing a large amount of data at one time.

[0077] Figure 3 This is a schematic diagram of the implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application. Figure 3 , the method can be executed by a processor of a computer device. Figure 2 The method further includes steps S301 to S302, combining Figure 3 The steps shown are explained.

[0078] Step S301: When the distribution of the keywords in the captured document fragment meets a preset condition, the sliding process is stopped.

[0079] The distribution of the above keywords in the document fragment includes, but is not limited to: the number of occurrences of the keywords in the document fragment, the position of the keywords in the document fragment, the contextual association between the keywords and the document fragment, the proportion of the keywords in the document fragment, etc. The preset conditions are set in advance according to the search requirements and are used to determine whether the document fragment meets the search requirements.

[0080] In some embodiments, during the process of capturing document segments by the sliding window, the distribution of keywords in each segment is monitored in real time and compared with a preset condition. Once the currently captured document segment meets the preset condition, the movement of the sliding window is stopped immediately.

[0081] In some embodiments, the preset condition includes at least one of the following: the intercepted document segment includes all keywords; the highlight value of the intercepted document segment satisfies a preset stop threshold.

[0082] For example, suppose the document to be highlighted is an article containing multiple paragraphs, and the keywords are "artificial intelligence" and "future". Under the preset condition of containing all keywords, when the sliding window intercepts a document segment that contains both the keywords "artificial intelligence" and "future", it stops sliding; under the preset condition of a highlight value of not less than 0.7, when the sliding window intercepts a document segment with a highlight value of 0.8, it stops sliding; under the preset condition of containing all keywords and a highlight value of not less than 0.7, when the sliding window intercepts a segment that contains both the keywords "artificial intelligence" and "future" and a highlight value of 0.75, it stops sliding.

[0083] Step S302: determine the captured document segment as the highlighted text selection.

[0084] In an embodiment of the present application, after the sliding window is stopped, the document fragment that currently meets the preset conditions is directly extracted and output as a highlighted text selection.

[0085] In an embodiment of the present application, in the process of capturing document fragments with a sliding window, when it is detected that the distribution of keywords in the document fragments meets preset conditions, the sliding process is stopped immediately, which can effectively avoid unnecessary resource consumption. At the same time, this also improves the accuracy and efficiency of information retrieval.

[0086] Figure 4 This is a schematic diagram of the implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application. Figure 4 , the method can be executed by a processor of a computer device. Figure 1 , the distribution condition includes at least one distribution parameter; Figure 1 Step S102 in can be updated to step S401 to step S402, combining Figure 4 The steps shown are explained.

[0087] Step S401: Determine the sub-value of each distribution parameter based on the internal keywords included in the document fragment.

[0088] The internal keyword refers to a keyword that appears in the document segment. The distribution parameters include at least one of the following: the number of categories of the internal keyword, the number of occurrences of the internal keyword, the length of the internal keyword that appears, and the position of the internal keyword in the document segment.

[0089] In some embodiments, the number of categories of internal keywords is the number of categories of internal keywords actually contained in the document fragment in at least one keyword obtained. It is understandable that the number of categories here is "kind" rather than "number", that is, multiple identical keywords are counted as only one category. For example, if the preset keywords are "artificial intelligence", "machine learning" and "deep learning", and "artificial intelligence" is mentioned twice and "machine learning" once in the document fragment, then the number of categories of internal keywords contained in the document fragment is 2 (although "artificial intelligence" appears twice, it is only counted as one category). In the case where the distribution parameter includes the number of categories of internal keywords, the sub-value of the number of categories can be determined by the following scheme: the more the number of internal keyword categories contained in the document fragment, the higher the correlation between the document fragment and the preset keywords may be, and therefore a higher sub-value can be given. Exemplarily, the calculation standard of the sub-value can be set based on the total categories (number) of the internal keywords obtained. For example, in the case where the total categories of internal keywords are 4, 0.25 points can be given for containing 1 keyword category, 0.5 points can be given for containing 2 keyword categories, 0.75 points can be given for containing 3 keyword categories, and 1 point can be given for containing 4 keyword categories, that is, full marks.

[0090] In some embodiments, when the distribution parameter includes the number of occurrences of the internal keyword, its corresponding sub-value can be determined by the following scheme: the more times the keyword appears, the higher the relevance of the document fragment to the keyword may be, that is, the higher the corresponding sub-value. Exemplarily, a threshold or range can be set to score according to the number of keyword appearances. For example, 1-2 occurrences give a lower score, 3-5 occurrences give a medium score, and more than 5 occurrences give a higher score. The specific score can be adjusted according to actual needs.

[0091] In some embodiments, when the distribution parameter includes the length of the internal keyword, its corresponding sub-value can be determined by the following scheme: shorter internal keywords may be more common, while longer internal keywords are often more specific and can better express the theme of the document fragment, that is, the sub-value corresponding to the shorter internal keyword is lower, and the sub-value corresponding to the longer internal keyword is higher. Exemplarily, different scores can be set for internal keywords of different lengths. For example, short keywords (such as 1-2 words) are given lower scores, medium-length keywords (such as 3-5 words) are given medium scores, and long keywords (such as more than 5 words) are given higher scores.

[0092] In some embodiments, when the distribution parameter includes the position of the internal keyword in the document fragment, its corresponding sub-value can be determined by the following scheme: the keyword appearing at the beginning or end of the document fragment is more likely to attract the reader's attention and is more likely to express the theme of the document fragment, that is, when the internal keyword is at the beginning or end of the document fragment, the corresponding sub-value is higher, and when the internal keyword is in the middle of the document fragment, the corresponding sub-value is lower. Exemplarily, a higher score can be set for the internal keyword at the beginning or end of the document fragment, and a lower score can be set for the internal keyword in the middle of the document fragment.

[0093] The above embodiment is described below using an actual scoring scenario: Assume that the keywords include artificial intelligence, machine learning, deep learning, and natural language processing, a total of 4, and the document snippet is: "With the rapid development of artificial intelligence technology, machine learning algorithms have been widely used in various fields. Deep learning, as an important branch of machine learning, is leading a new round of changes in artificial intelligence."

[0094] Under the distribution parameter of the number of categories of internal keywords, since the number of keyword categories contained in the document fragment is 3 (including artificial intelligence, machine learning, and deep learning), its sub-value calculation is: based on the total number of categories being 4, the scoring criteria are set as follows: 0.25 for containing 1 category, 0.5 for containing 2 categories, 0.75 for containing 3 categories, and 1 for containing 4 categories. Therefore, the sub-value of the document fragment in the number of categories of internal keywords is 0.75.

[0095] Under the distribution parameter of the number of occurrences of internal keywords, since the number of occurrences of internal keywords in the document fragment is artificial intelligence (2 times), machine learning (2 times), deep learning (1 time), and natural language processing (0 times); if the set scoring criteria are: 0 for 0 occurrences, 0.5 for 1 occurrence, 0.75 for 2 occurrences, and 1 for 3 occurrences or more; the scores of each keyword can be obtained: artificial intelligence (0.75), machine learning (0.75), deep learning (0.5), and natural language processing (0). After averaging, the sub-value of the number of occurrences of internal keywords in the document fragment is 0.5.

[0096] Under the distribution parameter of the length of internal keywords, the length of internal keywords in the document fragment is: artificial intelligence (4 words), machine learning (4 words), deep learning (4 words); if the scoring criteria are set as follows: short keywords (1-2 words) get 0.25, medium-length keywords (3-4 words) get 0.5, and long keywords (more than 5 words) get 1. Then we can get the score of each keyword: artificial intelligence (0.5), machine learning (0.5), deep learning (0.5), natural language processing (0). After averaging, the sub-value of the length of internal keywords in the document fragment is 0.375.

[0097] Under the distribution parameter of the position of internal keywords in the document fragment, the keyword positions in the document fragment are: artificial intelligence (beginning and end), machine learning (middle), deep learning (middle); if the scoring criteria are set as follows: 0.75 for appearing at the beginning or end, and 0.5 for appearing in the middle. Therefore, "machine learning" and "deep learning" each get 0.5 points, "artificial intelligence" gets 0.75 points, and "natural language processing" does not appear, so it gets 0 points. After averaging, the sub-value of the document fragment at the position of the document fragment is 0.4375.

[0098] In some embodiments, the above-mentioned determination of each sub-value of the distribution parameter based on the keywords included in the document fragment can be implemented through steps S4011 to S4012.

[0099] Step S4011: Obtain the word weight of each of the at least two keywords.

[0100] Among them, word weight is used to measure the importance of keywords in the document.

[0101] In some embodiments, the word weight of the keyword can be determined based on the frequency of the keyword appearing in the document to be highlighted. For example, if the keyword appears more times in the document to be highlighted, the more important the keyword is to the document, the greater the corresponding word weight.

[0102] In other embodiments, the word weight of the keyword can be determined based on the frequency of the keyword appearing in a preset document set (corpus). For example, if the keyword appears more times in the preset document set, that is, the more common it is in the document set, it means to a certain extent that the keyword is less important to the document, and the corresponding word weight is also smaller.

[0103] In some other embodiments, the above two situations may be combined, that is, the word weight of the keyword may be determined by taking into account the frequency of occurrence of the keyword in the document to be highlighted and the frequency of occurrence of the keyword in a preset document collection (corpus).

[0104] In some other embodiments, a machine learning algorithm (such as a support vector machine, a neural network) may be used to calculate the word weight for each keyword.

[0105] Step S4012: Determine the sub-value of each distribution parameter based on the internal keywords included in the document fragment and the word weights of the internal keywords.

[0106] In the embodiment of the present application, the word weight of the keyword can be increased on the basis of the implementation method provided in step S401, so as to obtain a sub-value of the distribution parameter that better meets the requirements. For a distribution parameter, the score of each internal keyword in the document fragment under the distribution parameter is obtained, and based on the word weight corresponding to each (internal) keyword, the score of each internal keyword under the distribution parameter is weighted and summed to obtain the sub-value of the distribution parameter.

[0107] Based on the above examples, the following examples use artificial intelligence (word weight is 0.8), machine learning (word weight is 0.8), deep learning (word weight is 0.5), and natural language processing (word weight is 0.4) as examples for explanation.

[0108] Under the distribution parameter of the number of occurrences of keywords, combined with the word weight of each keyword, the score of each keyword is: artificial intelligence (0.75*0.8=0.6), machine learning (0.75*0.8=0.6), deep learning (0.5*0.5=0.25), natural language processing (0*0.4=0), and thus, the obtained sub-value is (0.6+0.6+0.25+0) / 4=0.3625.

[0109] Under the distribution parameter of the length of the keywords that appear, combined with the word weight of each keyword, the score of each keyword is: artificial intelligence (0.5*0.8=0.4), machine learning (0.5*0.8=0.4), deep learning (0.5*0.5=0.25), natural language processing (0*0.4=0); and then, the obtained sub-value is (0.4+0.4+0.25+0) / 4=0.2625.

[0110] Under the distribution parameter of the position of the keyword in the document fragment, combined with the word weight of each keyword, the score of each keyword is: artificial intelligence (0.75*0.8=0.6), machine learning (0.5*0.8=0.4), deep learning (0.5*0.5=0.25), natural language processing (0*0.4=0); and then, the obtained sub-value is (0.6+0.4+0.25+0) / 4=0.3125.

[0111] Step S402: Determine a highlight value of the document segment based on the weight of each distribution parameter and each sub-value of the distribution parameter.

[0112] In the embodiment of the present application, the influence of multiple distribution parameters (such as the number of occurrences of keywords, the length of keywords, the position of keywords in the document fragment, etc.) on the document fragment score and the relative importance of these distribution parameters are comprehensively considered. By assigning weights to each distribution parameter and calculating the highlight value based on these weights and the corresponding sub-values, a comprehensive evaluation of the relevance of the document fragment can be achieved.

[0113] In some embodiments, the weight corresponding to each distribution parameter can be derived based on historical data, expert judgment or machine learning model, reflecting the importance of different distribution parameters in evaluating the relevance of document fragments. Then, according to the sub-values ​​of each distribution parameter calculated in step S401, these sub-values ​​are multiplied by their corresponding weights to obtain weighted sub-values. Finally, all weighted sub-values ​​are summed to obtain the highlight value of the document fragment.

[0114] In the embodiment of the present application, by determining the sub-values ​​corresponding to each distribution parameter (such as the number of keyword categories, the number of keyword occurrences, the length of keywords that appear, and the position of keywords in the document fragment) based on the keywords included in the document fragment, the matching degree of the document fragment with the keyword set in different dimensions can be carefully and comprehensively evaluated, thereby capturing the rich information of the document fragment. At the same time, by combining the weight and sub-value corresponding to each distribution parameter to determine the highlight value of the document fragment, the importance difference of different distribution parameters in evaluating the relevance of the document fragment can be fully considered, making the highlight value more accurate and in line with actual needs.

[0115] Figure 5 This is a schematic diagram of the implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application. Figure 5 , the method can be executed by a processor of a computer device. Figure 1 The method further comprises step S501, combining Figure 5 The steps shown are explained.

[0116] Step S501, displaying the highlighted text segment; wherein, the keywords in the highlighted text segment are displayed in a first state, and other texts in the highlighted text segment are displayed in a second state; the first state is different from the second state.

[0117] In some embodiments, the first state and the second state refer to different forms of text display, such as color, font, bold, underline, background highlight, etc., used to distinguish keywords from other text in the text. Of course, the other text is the non-keyword text in the highlighted text selection.

[0118] Exemplarily, based on the above example, in the process of displaying "With the rapid development of artificial intelligence technology, machine learning algorithms have been widely used in various fields. Deep learning, as an important branch of machine learning, is leading a new round of changes in artificial intelligence.", keywords such as "artificial intelligence", "machine learning" and "deep learning" can be displayed in the first state (such as bold, highlighted or using a special color font), while other parts of the text, such as "with the rapid development of...", "has been widely used in various fields" and "is leading a new round of changes in...", etc., remain in the second state (i.e., the regular text display state).

[0119] In the embodiment of the present application, the keywords are displayed in the first state (such as bold, highlighted or different colors), which can immediately attract the user's attention, allowing the user to quickly locate the core information in the text, and improve the efficiency and accuracy of information retrieval. At the same time, other texts are displayed in the second state as background information to support the context of the keywords, helping users to better understand the meaning and role of the keywords in the text. This differentiated display method not only improves the user experience, but also enhances the readability and comprehensibility of the text information.

[0120] Figure 6 This is a schematic diagram of the implementation process of a method for determining a highlighted text selection provided in an embodiment of the present application. Figure 6 , the method can be executed by a processor of a computer device. Figure 1 The method further includes steps S601 to S602, combining Figure 6 The steps shown are explained.

[0121] Step S601: When none of the keywords exists in the document to be highlighted, segment the at least one keyword to obtain at least two candidate keywords.

[0122] Among them, word segmentation refers to the process of dividing a continuous text string into independent vocabulary units. Alternative keywords refer to new words obtained through word segmentation operations that may exist in the document to be highlighted. These words can be used as substitutes for the original keywords to expand the search scope. When the keywords entered by the user cannot be found directly in the document to be highlighted, alternative keywords can be generated through word segmentation technology (that is, the keywords are broken down into smaller vocabulary units). These alternative keywords may appear alone or in combination in the document to be highlighted, thus serving as the basis for the search.

[0123] For example, suppose the keyword entered by the user is "artificial intelligence development", but the phrase is not found in the document to be highlighted. The word segmentation algorithm can be used to split "artificial intelligence development" into two candidate keywords "artificial intelligence" and "development", and then search in the document to be highlighted based on these two candidate keywords.

[0124] Step S602: Determine the highlighted text segment in the document to be highlighted based on the at least two candidate keywords.

[0125] In the embodiment of the present application, the above process of determining the highlighted text segment in the document to be highlighted based on the at least two candidate keywords is actually the same as Figure 1 In the embodiment, the scheme for determining a highlighted text segment in a document to be highlighted based on at least one keyword is similar, including: obtaining at least one document segment in the document to be highlighted; determining a highlight value of the obtained document segment based on the distribution of the candidate keywords in the document segment; and determining a highlighted text segment based on the document segment whose highlight value reaches a threshold and satisfies a threshold. The implementation details of the above process can refer to the above embodiment, and will not be repeated here.

[0126] In an embodiment of the present application, when any original keyword does not exist in the document to be highlighted, by performing word segmentation on at least one keyword, the search scope is expanded and the inclusiveness and accuracy of the search are improved. At the same time, based on these alternative keywords, the highlighted text selections in the document to be highlighted can be determined, and documents that are highly relevant to the user's query intent can be accurately found. Even if these documents do not directly use the original keywords, they can be effectively retrieved through the alternative keywords after word segmentation. This processing method not only enhances the flexibility of information retrieval, but also significantly improves the accuracy and relevance of the highlighted text selections, providing users with more accurate and comprehensive search results.

[0127] Figure 7 FIG. 1 is a schematic diagram of another implementation flow of a method for determining a highlighted text selection provided by an embodiment of the present application. The method can be executed by a processor of a computer device. Figure 7 The steps shown are explained.

[0128] Step S701: In response to receiving a highlighted text selection event for at least one keyword, display the highlighted text selection.

[0129] The highlighted text selection obtains a document segment whose highlight value reaches a threshold value in at least one document segment in the document to be highlighted; the highlight value is determined based on the distribution of the keyword in the document segment.

[0130] In the embodiment of the present application, the highlighted text selection event is an event that triggers the system to search for relevant text from the document to be highlighted when a user or system initiates a text search request for a specific keyword (at least one keyword). The keyword can be a word, phrase or string input by the user or selected by the system to describe the information requirement or content theme.

[0131] In some embodiments, at least one keyword related to the user's intent can be obtained through user input, system recommendation, or predefined methods for subsequent matching in documents. In some implementation scenarios, the device can provide a user input interface to allow the user to enter one or more keywords. In other embodiments, the device can also intelligently recommend one or more keywords based on the user's search history, behavior pattern, or context information.

[0132] In some embodiments, in response to the highlighted text selection event, the highlighted text selection may be obtained based on the following method: obtaining at least one keyword; obtaining at least one document segment in the document to be highlighted; determining the highlight value of the obtained document segment based on the distribution of the keyword in the document segment; and determining the highlighted text selection based on the document segment whose highlight value reaches a threshold. The implementation details of the above process can refer to the above embodiment, and will not be repeated here.

[0133] In some embodiments, the keywords in the highlighted text selection are displayed in a first state, and other text in the highlighted text selection are displayed in a second state; the first state is different from the second state.

[0134] In some embodiments, the first state and the second state refer to different forms of text display, such as color, font, bold, underline, background highlight, etc., used to distinguish keywords from other text in the text. Of course, the other text is the non-keyword text in the highlighted text selection.

[0135] Exemplarily, based on the above example, in the process of displaying "With the rapid development of artificial intelligence technology, machine learning algorithms have been widely used in various fields. Deep learning, as an important branch of machine learning, is leading a new round of changes in artificial intelligence.", keywords such as "artificial intelligence", "machine learning" and "deep learning" can be displayed in the first state (such as bold, highlighted or using a special color font), while other parts of the text, such as "with the rapid development of...", "has been widely used in various fields" and "is leading a new round of changes in...", etc., remain in the second state (i.e., the regular text display state).

[0136] In the embodiment of the present application, the keywords are displayed in the first state (such as bold, highlighted or different colors), which can immediately attract the user's attention, allowing the user to quickly locate the core information in the text, and improve the efficiency and accuracy of information retrieval. At the same time, other texts are displayed in the second state as background information to support the context of the keywords, helping users to better understand the meaning and role of the keywords in the text. This differentiated display method not only improves the user experience, but also enhances the readability and comprehensibility of the text information.

[0137] In an embodiment of the present application, by responding to receiving a highlighted text selection event for at least one keyword, the system can instantly trigger and display a highlighted text selection that is highly relevant to the target query, thereby improving the real-time nature of information retrieval and the user interaction experience; at the same time, since the highlighted text selection is obtained by calculating the highlight value based on the distribution of the keywords in the document fragment, this enables fast and accurate searching of the text content most relevant to the user query from a large number of documents, thereby optimizing the search experience.

[0138] The following describes the application of the highlighted text selection determination method provided in an embodiment of the present application in actual scenarios.

[0139] In some embodiments, the size of the sliding window is the length of the highlighted segment (fixed window size), and the sliding window moves backward (to the end of the text) from the starting position of the text. Each time it moves a position, the number of hits of the token (corresponding to the keywords in the above embodiment) in the current sliding window is counted to calculate the score of the sliding window at the current position. If the sliding window at the current position contains all the tokens, it stops early; otherwise, it continues to move backward until the end of the text, and then selects the position with the highest score based on the score of each slid position (corresponding to the highlight value in the above embodiment), and uses the paragraph determined by the sliding window at this position in the text as the highlighted segment of the current text.

[0140] See also Figure 8 , which shows a schematic diagram of the implementation of a sliding window algorithm. Figure 8 801 and 802 in the figure are different positions of the sliding window during the sliding process. At this time, the keywords to be highlighted are "search", "design" and "optimization". When the sliding window is at the position shown in 801, the sliding window includes the keywords "search" and "design". At this time, since it does not include all the keywords, it continues to move to the right. When it moves to 802, the sliding window includes all the keywords, so it stops moving.

[0141] In some embodiments, in the process of calculating the score at each position, the following implementation method can be used: obtain the text to be selected at the position of the sliding window, and count the keyword length, keyword occurrence number and keyword distribution in the text to be selected, and then score the position based on the keyword length, keyword occurrence number and keyword distribution.

[0142] It can be understood that the longer the keyword is, the higher the score of the position is; the more times the keyword appears, the higher the score of the position is; the more concentrated the distribution of the keywords is, the higher the score of the position is.

[0143] For example, if the keywords to be highlighted are "big data" and "machine learning", the following three paragraphs can be obtained based on these two keywords:

[0144] Paragraph 1: "With the advancement of technology and the rapid development of society, big data and machine learning are changing our world."

[0145] Paragraph 2: "Big data is an important foundation for machine learning. On this basis, machine learning also requires powerful computing power."

[0146] Paragraph 3: “Machine learning algorithms play a key role in big data processing, but users’ data privacy also needs to be considered.”

[0147] From the perspective of "the longer the keyword length, the higher the score", paragraph 1: "big data" (3 characters) and "machine learning" (4 characters) can get 3 points and 4 points, and the total length score is 7 points. The keywords in paragraphs 2 and 3 are the same as those in paragraph 1, so their scores on the feature of "keyword length" are also 7 points.

[0148] From the perspective of "the more times a keyword appears, the higher the score", paragraph 1: "Big data" and "machine learning" appear once each, with a total score of 1+1=2 points; paragraph 2: "Big data" appears once, "machine learning" appears twice, with a total score of 1+2=3 points. Paragraph 3: "Big data" and "machine learning" appear once each, with a total score of 1+1=2 points.

[0149] From the perspective of "the more concentrated the keyword distribution, the higher the score", paragraph 1: there is at least 2 characters between "big data" and "machine learning"; paragraph 2: there is at least 1 character between "big data" and "machine learning"; paragraph 3: there is at least 3 characters between "big data" and "machine learning". Therefore, paragraph 2 has the highest score and can be set to 3 points, paragraph 3 has the lowest score and can be set to 1 point, and paragraph 1's score is between paragraphs 2 and 3 and can be set to 2 points.

[0150] After obtaining the scores of the above three dimensions, the scores of each dimension can be normalized respectively. Then, for each paragraph, the normalized scores of the paragraph in each dimension are summed up to obtain the final score of each paragraph.

[0151] In the dimension of "keyword length", since all paragraphs have the same score, the scores of paragraphs 1 to 3 are all 1 after normalization; in the dimension of "number of keyword occurrences", the score of paragraph 1 is 0.67, the score of paragraph 2 is 1.00, and the score of paragraph 3 is 0.67; in the dimension of "more concentrated keyword distribution", the score of paragraph 1 is 0.67, the score of paragraph 2 is 1.00, and the score of paragraph 3 is 0.33. The normalized scores of each paragraph in each dimension are added together to obtain the final score.

[0152] Final score for paragraph 1: 1+0.67+0.67=2.34;

[0153] Final score for paragraph 2: 1+1+1=3.00;

[0154] Final score for paragraph 3: 1+0.67+0.33=2.00.

[0155] In order to further improve the accuracy of the highlighted selections, the above sliding window algorithm can be assisted based on a grading strategy. As the core of the entire algorithm, the sliding window can indeed efficiently locate potential related content when processing full-text searches. However, in complex and changeable actual scenarios, a single sliding window algorithm is often difficult to fully cover all needs, especially when faced with results of different matching degrees. Therefore, an embodiment of the present application provides a highlighted selection grading strategy, which aims to improve the accuracy and user experience of the highlighted selections through detailed classification and evaluation.

[0156] Based on the diversity and complexity of full-text search, the present embodiment of the application has carefully classified and graded the problems encountered. In the highlight selection algorithm, the present embodiment of the application divides the selection into the following three methods according to the matching degree:

[0157] Method 1 Best selection: This type of selection is a perfect match within the window, contains all the keywords (tokens) in the user's query, and is the highest-level highlight candidate.

[0158] Correspondingly, the method further includes: when the intercepted document segment includes all the keywords, stopping the sliding process, and using the document segment including all the keywords as the highlighted text selection.

[0159] Method 2: Secondary selection: Although this type of selection does not include all the keywords in the user's query, it still contains some keywords in the window, showing a high relevance. Although these selections are not the best, they meet the user's search needs to a certain extent.

[0160] Accordingly, the method further includes: when the intercepted document fragment includes at least two of the keywords, determining a highlight value of the acquired document fragment based on the distribution of the keywords in the document fragment. Subsequently, determining the highlighted text selection based on the document fragment whose highlight value reaches a threshold.

[0161] Method 3: Splicing selections: When a window contains only a single keyword, the embodiment of the present application adopts a splicing strategy to intelligently combine multiple window contexts containing a single keyword to form a coherent, highlighted selection containing more context information. Although this method increases the complexity of processing, it can significantly improve the comprehensiveness and depth of information obtained by users.

[0162] Correspondingly, the method further includes: when there is no document segment including at least two of the keywords in the document to be highlighted, splicing the segments to be spliced ​​of the keywords to obtain the highlighted text segment.

[0163] In order to better reflect the value of different selections, the embodiment of the present application sets corresponding grade scores for each level of selections, and these scores play an auxiliary role in the text relevance evaluation in the recall stage. Finally, the present application applies these scores to the sorting algorithm to improve the overall search effect. This can not only improve the accuracy of search results, but also enhance users' trust and satisfaction with search results.

[0164] The following are examples of three levels of highlighting. In this case, the keywords to be highlighted are "search", "design" and "optimization". Fig. 9 , which shows an example of a scene of the best selection. When the sliding window is located at position 901, the highlighted selection in the sliding window includes "search", "design" and "optimization" at the same time, so it can be used as the best selection in mode 1. Fig.10 , which shows a scene example diagram of the second-best segment. When the sliding window is at position 1001, the highlighted segment in the sliding window includes both "design" and "optimization"; when the sliding window is at position 1002, the highlighted segment in the sliding window includes both "design" and "optimization"; when the sliding window is at position 1003, the highlighted segment in the sliding window includes both "optimization" and "search"; therefore, the paragraphs corresponding to positions 1001 to 1003 can all be used as the second-best segment. Please refer to Fig.11, which shows an example of a scene of splicing selections. When the sliding window is located at position 1101, the highlighted selection in the sliding window includes "search"; when the sliding window is located at position 1102, the highlighted selection in the sliding window includes "design"; when the sliding window is located at position 1103, the highlighted selection in the sliding window includes "optimization"; the window at each position contains only one token, and the final highlighted selection will splice the contents of the three window segments together as a splicing selection in mode 3.

[0165] In text processing and search engine technology, highlighting is an important technology to enhance user experience, which allows users to intuitively identify the location of query keywords in the document. The following is an example of several common highlighters:

[0166] (1) Unified Highlighter: Unified Highlighter adopts a strategy that combines sentence segmentation with the BM25 scoring mechanism. First, the document is segmented into independent sentences using natural language processing (NLP) technology. Then, the BM25 algorithm is applied to each sentence for scoring. BM25 is an algorithm based on a probabilistic retrieval model that considers multiple factors such as term frequency (TF), inverse document frequency (IDF), and document length to calculate the relevance between terms and sentences. Finally, sentences with higher relevance are selected for highlighting based on the scoring results.

[0167] (2) Default highlighter (Plain Highlighter): Plain Highlighter is the default highlighter built into Elasticsearch (ES), which directly utilizes Lucene's standard highlighting mechanism. During query execution, PlainHighlighter creates a temporary index in memory and re-executes the original query to capture detailed matching information. This information includes the position and offset of the term in the document, which is used for subsequent highlighting.

[0168] (3) Fast Vector Highlighter (FVH) based on pre-built word vectors: FVH is an efficient highlighting solution. Its core lies in the fact that rich word vector information is pre-stored during the index construction phase. This word vector information includes the position, offset, and frequency of the word in the document, but does not involve complex embedding vectors. During the highlighting phase, FVH directly reads this pre-built vector information, quickly locates the matching items, and generates highlighting results without recalculating the matching position.

[0169] Each of the above highlighters has different principles and different applicable scenarios, but no highlighter can cover all scenarios (always weighing performance and matching degree). In addition, another important factor is that the extensions opened by these plug-ins are relatively limited. It is impossible to obtain the query recall strategy to adapt to the search and matching rules formulated at the application layer, and it is impossible to make in-depth adaptation with the business to improve the effect.

[0170] Based on the sliding window-based highlighting method provided in the above embodiment, the embodiment of the present application focuses more on the long text search scenario. In the optimization process of full-text search, the highlighting logic is separated from ES and a highlighting system is implemented externally. At the same time, a set of implicit word segmentation and matching rule standards are formulated. There are different recall and word segmentation rules for different query text types, as well as space segmentation and punctuation segmentation of the query. In turn, there will be different hierarchical DSLs (query statements) that have a strict correspondence with the token phrases in the highlighting stage. At the same time, there is a corresponding query token degradation strategy in the highlighting stage, supplemented by a highlighting hierarchical strategy (the query token degradation strategy and the highlighting hierarchical strategy are different strategies), forming a complete highlighting system suitable for long text search scenarios such as full text.

[0171] Based on the foregoing embodiments, an embodiment of the present application provides a device for determining a highlighted text selection, which includes the various units included and the various modules included in the units, and can be implemented by a processor in a computer device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0172] Fig.12 A schematic diagram of the structure of a device for determining a highlighted text selection provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the highlighted text selection determining device 1200 includes: a first acquisition module 1210, a second acquisition module 1220, a first determination module 1230, and a second determination module 1240, wherein:

[0173] A first acquisition module 1210, configured to acquire at least one keyword;

[0174] A second acquisition module 1220 is used to acquire at least one document segment in the document to be highlighted;

[0175] A first determination module 1230, configured to determine a highlight value of the acquired document fragment based on the distribution of the keyword in the document fragment;

[0176] The second determination module 1240 is configured to determine a highlighted text segment based on the document segment whose highlight value reaches a threshold.

[0177] In some embodiments, the second acquisition module 1220 is further used to: use a sliding window to slide from a starting position in the document to be highlighted; during the sliding process, intercept the document segment in the document to be highlighted based on the sliding window.

[0178] In some embodiments, the first determination module 1230 is further used to: in response to intercepting the document fragment, determine the highlight value of the acquired document fragment based on the distribution of the keywords in the document fragment.

[0179] In some embodiments, the second determination module 1240 is further used to: stop the sliding process when the distribution of the keywords in the captured document fragment meets a preset condition; and determine the captured document fragment as the highlighted text selection.

[0180] In some embodiments, the distribution condition includes at least one distribution parameter; the first determination module 1230 is also used to: determine the sub-value of each distribution parameter based on the internal keywords included in the document fragment; determine the highlight value of the document fragment based on the weight of each distribution parameter and the sub-value of each distribution parameter.

[0181] In some embodiments, when there are at least two keywords, the first determination module 1230 is also used to: obtain the word weight of each of the at least two keywords; and determine the sub-value of each distribution parameter based on the internal keywords included in the document fragment and the word weights of the internal keywords.

[0182] In some embodiments, the document fragment whose highlight value reaches the threshold includes the fragment to be spliced ​​of each of the keywords; the second determination module 1240 is also used to: when there is no document fragment including at least two of the keywords in the document to be highlighted, splice the fragments to be spliced ​​of each of the keywords to obtain the highlighted text selection.

[0183] In some embodiments, the highlighted text selection determining device 1200 includes a text display module, which is used to display the highlighted text selection; wherein the keywords in the highlighted text selection are displayed in a first state, and other texts in the highlighted text selection are displayed in a second state; the first state is different from the second state.

[0184] In some embodiments, the highlighted text selection determining device 1200 includes a word segmentation module, which is used to segment the at least one keyword to obtain at least two alternative keywords when none of the keywords are present in the document to be highlighted; and determine the highlighted text selection in the document to be highlighted based on the at least two alternative keywords.

[0185] Fig.13 A schematic diagram of the structure of another device for determining a highlighted text selection provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the highlighted text selection determining device 1300 includes: a display module 1310, wherein:

[0186] The display module 1310 is used to display a highlighted text selection in response to receiving a highlighted text selection event for at least one keyword; wherein the highlighted text selection is determined based on the document segment whose highlight value reaches a threshold in the document to be highlighted; and the highlight value is determined based on the distribution of the keyword in the document segment.

[0187] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0188] It should be noted that in the embodiment of the present application, if the above-mentioned highlighted text selection determination method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0189] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0190] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0191] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0192] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0193] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0194] Fig.14 A hardware entity diagram of a computer device provided in an embodiment of the present application is shown in FIG. Fig.14 As shown, the hardware entity of the computer device 1400 includes: a processor 1401 and a memory 1402, wherein the memory 1402 stores a computer program that can be run on the processor 1401, and the processor 1401 implements the steps in the method of any of the above embodiments when executing the program.

[0195] The memory 1402 stores computer programs that can be run on the processor. The memory 1402 is configured to store instructions and applications executable by the processor 1401. It can also cache data to be processed or processed by the processor 1401 and various modules in the computer device 1400 (for example, image data, audio data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0196] When the processor 1401 executes the program, the steps of any of the above-mentioned methods for determining a highlighted text selection are implemented. The processor 1401 generally controls the overall operation of the computer device 1400 .

[0197] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the method for determining a highlighted text selection in any of the above embodiments.

[0198] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0199] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0200] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0201] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0202] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0203] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0204] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0205] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the above storage medium includes: mobile storage devices, read-only memory (ROM), disks or optical disks, etc. Various media that can store program codes.

[0206] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0207] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for determining a highlighted text segment, characterized in that: The method comprises: Get at least one keyword; Obtain at least one document fragment in the document to be highlighted; Determining a highlight value of the acquired document fragment based on the distribution of the keyword in the document fragment; Based on the document segments whose highlight values ​​reach a threshold, a highlighted text selection is determined.

2. The method according to claim 1, characterized in that The obtaining of at least one document fragment in the document to be highlighted includes: Using a sliding window to slide from a starting position in the document to be highlighted; During the sliding process, the document segment is intercepted in the document to be highlighted based on the sliding window.

3. The method according to claim 2, characterized in that The step of determining the highlight value of the acquired document fragment based on the distribution of the keyword in the document fragment includes: In response to intercepting the document segment, a highlight value of the acquired document segment is determined based on the distribution of the keywords in the document segment.

4. The method according to claim 3, characterized in that The method further comprises: When the distribution of the keywords in the intercepted document fragment meets the preset conditions, the sliding process is stopped; The intercepted document segment is determined as the highlighted text segment.

5. The method according to any one of claims 1 to 4, characterized in that: The distribution condition includes at least one distribution parameter; and determining the highlight value of the acquired document fragment based on the distribution condition of the keyword in the document fragment includes: Determining a sub-value of each of the distribution parameters based on internal keywords included in the document fragment; A highlight value of the document fragment is determined based on the weight of each of the distribution parameters and the sub-value of each of the distribution parameters.

6. The method according to claim 5, characterized in that In the case where there are at least two keywords, the step of determining the sub-value of each distribution parameter based on the internal keywords included in the document fragment comprises: Obtaining a word weight of each of the at least two keywords; Based on the internal keywords included in the document fragment and the word weights of the internal keywords, the sub-values ​​of each of the distribution parameters are determined respectively.

7. The method according to any one of claims 1 to 6, characterized in that: The document segments whose highlight values ​​reach the threshold value include segments to be spliced ​​of each of the keywords; and determining the highlighted text selection based on the document segments whose highlight values ​​reach the threshold value includes: In the case that there is no document segment including at least two of the keywords in the document to be highlighted, the segments to be spliced ​​of the keywords are spliced ​​to obtain the highlighted text segment.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The highlighted text selection is displayed; wherein, the keywords in the highlighted text selection are displayed in a first state, and other texts in the highlighted text selection are displayed in a second state; the first state is different from the second state.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: When none of the keywords exists in the document to be highlighted, segmenting the at least one keyword to obtain at least two candidate keywords; The highlighted text segment is determined in the document to be highlighted based on the at least two candidate keywords.

10. A method for determining a highlighted text segment, characterized in that: The method comprises: In response to receiving a highlighted text selection event for at least one keyword, displaying the highlighted text selection; The highlighted text segment is determined based on the document segment whose highlight value reaches a threshold in the document to be highlighted; and the highlight value is determined based on the distribution of the keyword in the document segment.

11. A device for determining a highlighted text segment, characterized in that: The device comprises: A first acquisition module, used to acquire at least one keyword; A second acquisition module is used to acquire at least one document fragment in the document to be highlighted; A first determination module, configured to determine a highlight value of the acquired document fragment based on the distribution of the keyword in the document fragment; The second determination module is used to determine a highlighted text segment based on the document segment whose highlight value reaches a threshold.

12. A device for determining a highlighted text segment, characterized in that: The device comprises: A display module, configured to display a highlighted text selection in response to receiving a highlighted text selection event for at least one keyword; The highlighted text segment is determined based on the document segment whose highlight value reaches a threshold in the document to be highlighted; and the highlight value is determined based on the distribution of the keyword in the document segment.

13. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented, or the steps of the method described in claim 10 are implemented.

Images