Content search method, device, equipment, and medium

Abstract

The present application discloses a content search method, apparatus, device, and medium, relating to the field of text processing technology. The method comprises the following steps: obtaining search condition content; when the length of the search condition content indicates that the search condition content belongs to sentence content, performing attention analysis on the search condition content to determine a first keyword set of the search condition content; performing vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary, and a second keyword set including the first keyword set and the expanded vocabulary; performing feature extraction on the second keyword set to obtain a keyword feature representation; and based on the degree of match between the keyword feature representation and candidate content in a candidate content library, obtaining at least one first content from the candidate content library as a search result of the search condition content. This method can improve the degree of match between the search result and the search condition.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of text processing technology, and in particular to a content search method, apparatus, device, and medium. Background Art

[0002] The search system provides users with search results based on the search criteria they enter, helping them quickly retrieve desired content, such as writing materials, information articles, etc.

[0003] In related technologies, users input text content as search conditions, and the search system analyzes the search conditions, determines the keywords in the search conditions, matches the keywords with various contents in the database, and feeds back to the user the matching results that meet the preset requirements.

[0004] However, the matching method based on surface keywords does not take into account the context of the keywords, cannot distinguish ambiguity, puns, etc. in natural language, and has insufficient analysis of the context, resulting in low accuracy of matching results and poor content feedback to users. Summary of the Invention

[0005] The embodiments of the present application provide a content search method, apparatus, device, and medium that can improve the matching degree of search results. The technical solution is as follows:

[0006] In one aspect, a content search method is provided, the method comprising:

[0007] Acquire search condition content, where the search condition content is content input by a user for searching;

[0008] When the length of the search condition content indicates that the search condition content is a sentence content, performing attention analysis on the search condition content to determine a first keyword set for the search condition content, the first keyword set including at least one keyword in the search condition content, the at least one keyword being a word in the search condition content that meets a preset attention condition;

[0009] performing vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary, and a second keyword set including the first keyword set and the expanded vocabulary, wherein the expanded vocabulary meets a preset correlation requirement with the keywords in the first keyword set;

[0010] performing feature extraction on the second keyword set to obtain keyword feature representation;

[0011] Based on the matching degree between the keyword feature representation and the candidate content in the candidate content library, at least one first content is obtained from the candidate content library as a search result of the search condition content.

[0012] In another aspect, a content search device is provided, the device comprising:

[0013] An acquisition module is used to acquire search condition content, where the search condition content is content input by a user for searching;

[0014] an attention analysis module, configured to, when the length of the search condition content indicates that the search condition content is sentence content, perform attention analysis on the search condition content to determine a first keyword set for the search condition content, the first keyword set including at least one keyword in the search condition content, the at least one keyword being a word in the search condition content that meets a preset attention condition;

[0015] a vocabulary expansion module configured to perform vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary, and a second keyword set comprising the first keyword set and the expanded vocabulary, wherein the expanded vocabulary meets a preset correlation requirement with the keywords in the first keyword set;

[0016] a feature extraction module, configured to extract features from the second keyword set to obtain keyword feature representations;

[0017] The search result determination module is configured to obtain at least one first content from the candidate content library as a search result of the search condition content based on a matching degree between the keyword feature representation and the candidate content in the candidate content library.

[0018] In an optional embodiment, the attention analysis module is further used to perform word segmentation processing on the search condition content to obtain a word sequence arranged in order after the segmentation, and each word in the word sequence corresponds to a serial number; the word sequence is processed by multiple transformer layers in the deep learning model, and a self-attention weight matrix is ​​output, wherein the i-th row element in the self-attention weight matrix is ​​used to indicate the attention distribution of the word with serial number i, and the j-th column element in the self-attention weight matrix is ​​used to indicate the attention weight assigned to each word in the word sequence to the word with serial number j, and the i-th row and j-th column element in the self-attention weight matrix is ​​used to indicate the attention weight assigned to each word in the word sequence to the word with serial number j. The element represents the attention weight assigned by the word with sequence number i to the word with sequence number j in the word sequence; i and j are positive integers; the self-attention weight matrix is ​​adjusted to obtain a weighted self-attention weight matrix; the attention score of each word in the word sequence is determined based on the weighted self-attention weight matrix, and the attention score is used to indicate the importance of each word in the word sequence in the search condition content; the first keyword set of the search condition content is determined based on the attention score of each word in the word sequence, wherein the attention score of the words in the first keyword set is greater than a preset score threshold.

[0019] In an optional embodiment, the attention analysis module is further used to perform feature extraction on the search condition content to obtain a first sentence feature representation; and determine a weighted self-attention weight matrix based on the product between the first sentence feature representation and the self-attention weight matrix.

[0020] In an optional embodiment, the vocabulary expansion module is further used to perform feature extraction on the at least one keyword in the first keyword set to obtain at least one corresponding word feature representation; obtain a vocabulary and a feature representation library corresponding to the vocabulary, and there is a mapping relationship between the multiple word feature representations in the feature representation library and the multiple words in the vocabulary, wherein the kth word feature representation in the feature representation library is obtained after feature extraction of the kth word in the vocabulary, and k is a positive integer; determine the synonym feature representation corresponding to the at least one word feature representation in the feature representation library; determine the corresponding synonym in the vocabulary as the extended vocabulary based on the synonym feature representation, wherein the semantics of the synonym and the semantics of the at least one keyword meet the preset similarity requirements.

[0021] In an optional embodiment, the vocabulary expansion module is further used to calculate the similarity between the mth word feature representation in the at least one word feature representation and the multiple word feature representations in the feature representation library, and determine the word feature representation whose similarity reaches a preset threshold as the synonym feature representation corresponding to the mth word feature representation; m is a positive integer.

[0022] In an optional embodiment, the feature extraction module is further configured to extract features from the search condition content to obtain a feature representation of the first sentence;

[0023] The device further comprises:

[0024] a vocabulary screening module, configured to process the first sentence feature representation and the keyword feature representation using a text sorting algorithm to obtain a semantic similarity image, wherein the semantic similarity image includes a main node representing the search condition content and multiple sub-nodes representing the vocabulary in the second keyword set, wherein multiple links between the main node and the multiple sub-nodes respectively correspond to node weights, and the node weights are used to indicate the semantic similarity between the vocabulary in the second keyword set and the search condition content; iteratively calculate and update the node weights in the semantic similarity image using the text sorting algorithm; and when the number of iterations reaches a preset iteration number threshold, screen the second keyword set based on the node weights to obtain a third keyword set;

[0025] The feature extraction module is further configured to perform feature extraction on the third keyword set to obtain the keyword feature representation.

[0026] In an optional embodiment, the search result determination module is further used to extract features from the candidate content in the candidate content library to obtain a feature representation pool corresponding to the candidate content library, the candidate content belongs to sentence content, and the nth sentence feature representation in the feature representation pool has a mapping relationship with the nth sentence content in the candidate content, where n is a positive integer; based on the similarity between the keyword feature representation and the sentence feature representation in the feature representation pool, a candidate content list is determined from the candidate content library, and the candidate content list includes multiple first contents; based on the matching degree between the first sentence feature representation and the feature representations of the multiple first contents, the at least one first content is determined from the candidate content list as the search result of the search condition content.

[0027] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a content search method as described in any of the above-mentioned embodiments of the present application.

[0028] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a content search method as described in any of the above-mentioned embodiments of the present application.

[0029] In another aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the content search method described in any of the above embodiments.

[0030] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0031] By analyzing the search terms entered by the user and determining the amount of attention each word receives from the rest of the search terms within a sentence, the system determines the importance of each word within the sentence and selects keywords as the basis for expansion. Expanding keywords based on context allows for a more accurate understanding of the semantics of the search terms based on the expanded terms, taking into account potential ambiguity and double meanings within the text. Compared to related techniques that rely solely on surface keyword matching to provide search results, this method improves the accuracy of matching results and provides results that better meet the search requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 is a schematic diagram of a content search system provided by an exemplary embodiment of the present application;

[0034] Figure 2is a flow chart of a content search method provided by an exemplary embodiment of the present application;

[0035] Figure 3 is a flowchart of a content search method provided by another exemplary embodiment of the present application;

[0036] Figure 4 is a structural block diagram of a content search device provided by an exemplary embodiment of the present application;

[0037] Figure 5 is a structural block diagram of a content search device provided by another exemplary embodiment of the present application;

[0038] Figure 6 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0040] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0041] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0042] It should be noted that the information and data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0043] It should be understood that although the terms first, second, etc. may be used in this application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0044] First, a brief introduction to the terms involved in the embodiments of this application is given:

[0045] Word2Vec (Word to Vector) model: Its main purpose is to generate word vectors, that is, to map words into a vector space. Each word is represented as a vector of fixed dimension, and these vectors can capture the semantic relationship between words.

[0046] In this application, feature representation refers to quantity. The process of extracting features from text to obtain feature representation can be regarded as the process of converting text into a vector. Feature extraction of vocabulary is performed to obtain word vectors (i.e., word feature representation), and feature extraction of sentences is performed to obtain sentence vectors (sentence feature representation).

[0047] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on the Transformer architecture. It also converts text into vectors. Compared to the Word2Vector model, the BERT model generates contextual word vectors. It uses a multi-layer self-attention mechanism to simultaneously consider contextual information on the left and right sides of a word, generating deep bidirectional representations.

[0048] Among them, the deep learning model for processing word sequences in this application refers to the BERT model.

[0049] The TextRank algorithm (Text Ranking Algorithm) is a graph-based ranking algorithm that segments text into sentences and performs word segmentation, using statistical metrics to assess the importance of words or phrases. It uses the co-occurrence relationships between words or phrases to construct a graph, where nodes represent words or sentences, and edge weights represent their similarity or co-occurrence frequency. By iteratively calculating the importance scores of nodes in the graph until convergence, the ranking of keywords or sentences is ultimately achieved.

[0050] In this application, a text sorting algorithm is used to sort the words in the search content condition and the second keyword set (including the words in the first keyword set and the extended words) to determine several keywords that are most similar in semantics to the search content condition.

[0051] The search system is an efficient tool that can help users quickly retrieve the information they want based on the search conditions entered by the users.

[0052] In related technologies, the system analyzes the text content input by the user, identifies and extracts key words, and then accurately matches them with the content in the database, presenting the content that meets the preset standards as search results to the user, thereby improving the efficiency and accuracy of user retrieval.

[0053] However, relying solely on keyword matching has certain limitations. The system matches content based on the literal meaning of keywords, ignoring the specific context in which the keywords appear. This makes it ineffective in identifying and distinguishing common phenomena in natural language, such as ambiguity and puns. Without contextual analysis, the system may fail to accurately grasp the user's true intent, resulting in low accuracy in matching results and search results that do not meet user expectations.

[0054] The present application provides a content search method that can reasonably expand the search criteria, analyze the proportion of attention each word in the search criteria receives from other words, determine the important keywords in the search criteria in combination with the context, and correctly understand the semantics of the search criteria when the search criteria belong to the sentence content (that is, the search criteria are of high dimension). Vocabulary expansion is performed based on keywords, and content that meets the search criteria is matched to the expanded vocabulary as search results for feedback. It can appropriately expand on the basis of correctly understanding the text semantics of the search criteria, and determine the content with a high matching degree to present to the user as the search result.

[0055] Figure 1 is a schematic diagram of a content search system provided by an exemplary embodiment of the present application, wherein the content search system can meet the user's needs for searching based on high-dimensional search conditions and low-dimensional search conditions.

[0056] The content search system 100 involves a user terminal 110 and a server terminal 120 , and the user terminal 110 and the server terminal 120 are connected via a communication network 130 .

[0057] The user terminal 110 inputs search condition content and sends it to the server terminal 120 . The server terminal 120 analyzes the search condition content, determines the search results, and feeds the search results back to the user terminal 110 .

[0058] Optionally, when the server 120 receives the search condition content sent by the user 110, it first performs a dimension judgment on the search condition content. Wherein, the search condition content is text content, and this embodiment performs the dimension judgment based on Chinese characters.

[0059] If the number of Chinese characters contained in the search condition content does not reach 5, it is determined that the search condition content belongs to low-dimensional content, that is, the search condition is determined to be vocabulary content; if the number of Chinese characters contained in the search condition content reaches 5, it is determined that the search condition content belongs to high-dimensional content, that is, the search condition is determined to be sentence content.

[0060] (1) When the search condition is a sentence content, the search condition content is split into multiple words, and the attention mechanism of the BERT model is used to analyze the search condition content. The amount of attention allocated to each word in the search condition content from other words is determined based on the self-attention weight matrix of each transformer layer in the BERT model, and the attention of each word is normalized. The attention weight ratio obtained after normalization is used as a benchmark to rank the importance of each word, and some words with higher rankings are selected as keywords to obtain a first keyword set. The first keyword set is used for vocabulary expansion to obtain expanded words with similar or identical semantics to the first keyword set, and the expanded words and the first keyword set are combined to form a second keyword set.

[0061] Feature extraction is performed on the second keyword set and the search condition content respectively to obtain word feature representations corresponding to each word in the second keyword set and the first sentence feature representation corresponding to the search condition content.

[0062] The server side 120 obtains a preset feature representation pool in the preparation stage (i.e., the stage before receiving the search condition content), in which there are multiple sentence feature representations, each of which is obtained by mapping a piece of text content (sentence text content).

[0063] The similarity between the sentence feature representations in the feature representation pool and the word feature representations corresponding to the second keyword set is calculated one by one, and the sentence feature representations with the highest similarity are used as candidate feature representations. The candidate feature representations are matched with the first sentence feature representation, and the text content corresponding to the candidate feature representation with the highest matching degree is fed back to the user terminal 110 as the search result.

[0064] (2) When the search condition is a vocabulary content, the search condition is firstly corrected for homophone errors, and homophone typos in the search condition are corrected to obtain a corrected vocabulary. The corrected vocabulary is then associated with keywords to obtain an expanded vocabulary. The expanded vocabulary is combined with the corrected vocabulary to obtain an expanded vocabulary set.

[0065] Feature extraction is performed on the extended vocabulary set to obtain a corresponding word feature representation set. That is, the word feature representation set is the result of mapping the extended vocabulary set in the vector space. Based on the distance between the feature representations, several word feature representations closest to the word feature representation set are determined from the vector space to obtain synonyms of the extended vocabulary set, and the synonyms are added to the extended vocabulary set.

[0066] After obtaining the extended vocabulary set, the sentence feature representations in the feature representation pool are calculated one by one with the word feature representations corresponding to the extended vocabulary set. Several sentence feature representations with the highest similarity are used as candidate feature representations. Matching is performed based on the candidate feature representations and the feature representations corresponding to the search condition content. The text content corresponding to the candidate feature representation with the highest matching degree is fed back to the user terminal 110 as the search result.

[0067] To sum up, the content search system and content search method in this application can simultaneously deal with search condition content of different dimensions, reasonably expand the search condition content, fully understand the semantics of the search condition content in combination with the context, and then feedback the search results, thereby improving the accuracy of the search results.

[0068] Combined with the above-mentioned noun introduction and application scenarios, the content search method provided by this application is described. The method can be executed by a server or a terminal, or by both a server and a terminal. In the embodiment of this application, the method is described as being executed by a terminal. Figure 2 As shown, Figure 2 FIG1 is a flowchart of a content search method provided by an exemplary embodiment of the present application. The method includes the following steps.

[0069] Step 210: Obtain search condition content.

[0070] The search condition content is the content input by the user for searching. The search condition content is text content with a certain length, including but not limited to characters, punctuation marks, numbers, etc.

[0071] For example, the search condition content is as follows: "human health report", which contains 6 characters, indicating that the user expects to retrieve information related to human health report.

[0072] Step 220 : When the length of the search condition content indicates that the search condition content belongs to sentence content, attention analysis is performed on the search condition content to determine a first keyword set for the search condition content.

[0073] After obtaining the search condition content, the dimension of the search condition content is first determined.

[0074] Users can enter words, sentence fragments, sentences, etc. of any dimension as search condition content, and judge based on the length of the search condition. If the length is less than 5, it is judged as a word; if the length is greater than or equal to 5, it is judged as a sentence or sentence fragment. Hereinafter, sentences or sentence fragments are referred to as sentence content.

[0075] In some embodiments, a vocabulary is prepared in advance, which contains a variety of words, including search conditions whose user input frequency reaches a preset frequency threshold (such as 5 searches out of every 100 searches contain the same word), associated search conditions (words with a high degree of association determined by combining the search conditions entered by the user within a historical time period), etc. When judging the dimension of the search condition content, the dimension of the search condition can also be determined based on whether the search condition content is included in the vocabulary. For example, if the length of the search condition content is greater than or equal to 5 and is not in the constructed vocabulary, the search condition content is judged to be sentence content.

[0076] For example, if the search condition content is as follows: "health, diet, recipes", the length of the search condition content is based on the number of Chinese characters, and if it contains 6 Chinese characters, the search condition content is a sentence content.

[0077] For example, if the search condition content is as follows: "health, recipe", the length of the search condition content is based on the number of Chinese characters, and if it contains 4 Chinese characters, then the search condition content is vocabulary content.

[0078] Optionally, the first keyword set includes at least one keyword in the search condition content, and the at least one keyword refers to a word in the search condition content that meets a preset attention condition.

[0079] The search condition content is segmented to obtain a sequence of words arranged in order after the segmentation, and each word in the sequence has a corresponding serial number.

[0080] For example, the search condition is: "Cultivation methods of desert plants". After splitting, the word sequence obtained is as follows: "desert", "plant", "of", "cultivation", "method", and the sequence numbers are 1 to 5.

[0081] The word sequence is processed through multiple transformer layers in a deep learning model (BERT model), and the output is a self-attention weight matrix. The elements in the i-th row of the self-attention weight matrix are used to indicate the attention distribution of the word with sequence number i, and the elements in the j-th column of the self-attention weight matrix are used to indicate the attention weight assigned by each word in the word sequence to the word with sequence number j. The elements in the i-th row and j-th column of the self-attention weight matrix represent the attention weight assigned by the word with sequence number i to the word with sequence number j in the word sequence. i and j are positive integers.

[0082] The self-attention weight matrix is ​​adjusted to obtain the weighted self-attention weight matrix.

[0083] The search condition content is feature extracted to obtain the first sentence feature representation, and a weighted self-attention weight matrix is ​​determined based on the product of the first sentence feature representation and the self-attention weight matrix.

[0084] The attention score of each word in the word sequence is determined based on the weighted self-attention weight matrix. The attention score is used to indicate the importance of each word in the word sequence in the search condition content.

[0085] A first keyword set of the search condition content is determined based on the attention score of each word in the word sequence, wherein the attention score of the words in the first keyword set is greater than a preset score threshold.

[0086] For example, the search criteria are split into the following sequence [t1, t2, ..., ta] and fed into the BERT model. Each tb (vocabulary) is tokenized and converted into a sequence of tokens that the BERT model can understand, where a is a positive integer and b is a positive integer not exceeding a. A special token [CLS] is added at the beginning of the sequence to aggregate sentence-level semantic information. A token [SEP] is added at the end of the sequence to indicate the end of a sentence.

[0087] The tokenization result of the input sequence is [101, t1, t2...ta, 102], where 101 and 102 are the tag IDs (identifiers) of [CLS] and [SEP] respectively.

[0088] The output of the BERT model is a vector matrix that contains the feature representation of each token ID in the sentence. These feature representations are used to represent the hidden state of each token in the sentence in the last layer of the model.

[0089] Each tag ID is mapped to the corresponding word feature representation through the BERT model, and the embedding matrix E is obtained with a size of (a+2)×d, where d is the dimension of the word feature representation.

[0090] There are several ways to obtain the first sentence feature representation that can represent the entire search condition content from the output of the BERT model.

[0091] (1) Obtain the feature representation corresponding to the [CLS] tag as the first sentence feature representation: The first sentence feature representation is S, S = E[1,:]. “E[1,:]” is used to access all columns of the second row of the matrix E. “1” indicates selecting the second row (the index usually starts at 0, and the [CLS] tag in the matrix has an index of 0), and “:” indicates selecting all columns of this row.

[0092] (2) Aggregate all word feature representations (e.g., by average pooling) to obtain the first sentence feature representation: S = mean(E[1:a+1,:], 1), ignoring the [SEP] tag. "[1:a+1,:]" is used to select all columns from the 1st row to the a+1th row from the matrix E. Here a is a specific integer representing the number of tags in the sequence (excluding the end-of-sequence tag). "[1:a+1]" includes the rows from the first tag to the a+1th tag, and ":" indicates that all columns of these rows are selected. The mean function is used to calculate the average value of the matrix E in the specified dimension. The parameter "1" in mean(……, 1) indicates that the operation is performed along the second dimension (column) of the matrix E, that is, the elements of each row are averaged.

[0093] For example, the output weights of the self-attention layer are extracted from each Transformer layer of the BERT model, and for each tb (vocabulary), the weighted sum of the self-attention weights of all layers is calculated as the attention score of each vocabulary in the word sequence.

[0094] Assume that the self-attention weight matrix of the lth (lowercase L) layer Transformer is A^l, with a size of (a+2)×(a+2). After multiplying the first sentence vector by the self-attention weight matrix, we get the weighted self-attention weight matrix.

[0095] On this basis, for each word tb, the weighted sum of its self-attention weights in all layers (i.e., attention score) can be expressed as: importance(tb) = sum(l = 1 to L, sum(A^l[:, i])), where L is the number of Transformer layers.

[0096] Apply the Softmax normalization method, convert the attention score of each word into a percentage and sort them, and take the words with the highest scores as the first keyword set.

[0097] For example, the search condition content is divided into 10 words, and the three words with the highest scores are determined as the first keyword set.

[0098] Step 230 : Perform vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary and a second keyword set including the first keyword set and the expanded vocabulary.

[0099] The expanded vocabulary and the keywords in the first keyword set meet the preset correlation requirements.

[0100] Optionally, feature extraction is performed on at least one keyword in the first keyword set to obtain at least one corresponding word feature representation.

[0101] A vocabulary and a feature representation library corresponding to the vocabulary are obtained, where a mapping relationship exists between multiple word feature representations in the feature representation library and multiple words in the vocabulary, wherein the k-th word feature representation in the feature representation library is obtained after feature extraction of the k-th word in the vocabulary, and k is a positive integer.

[0102] A synonym feature representation corresponding to at least one word feature representation is determined in the feature representation library.

[0103] For the mth word feature representation in the at least one word feature representation, similarities are calculated between the mth word feature representation and multiple word feature representations in the feature representation library, and word feature representations whose similarities reach a preset threshold are determined as synonym feature representations corresponding to the mth word feature representation. m is a positive integer.

[0104] Based on the synonym feature representation, corresponding synonyms are determined in the vocabulary as extended vocabulary, where the semantics of the synonyms meet a preset similarity requirement with the semantics of at least one keyword. The preset similarity requirement means that, for each of the at least one keyword, the vocabulary with the closest vector distance to the keyword is selected as a synonym for the keyword.

[0105] For example, the first keyword set includes two keywords: "fitness" and "recipe". After feature extraction, the corresponding word feature representations are C1 (1, 0, 4, 5) and C2 (5, 2, 1, 3). The feature representation library includes word feature representations K1, K2...K909.

[0106] The vector distances between C1 and K1, K2...K909 are calculated respectively, and it is determined that the vector distance between K501 (1, 0, 4, 4) and C1 is the smallest. Therefore, the similarity between K501 and C1 is the highest, which is the synonym feature representation T1; the vector distances between C2 and K1, K2...K909 are calculated respectively, and the vector distance between K128 (5, 2, 1, 4) and C2 is the smallest. Therefore, the similarity between K128 and C2 is the highest, which is the synonym feature representation T2.

[0107] Among them, the synonym feature indicates that the vocabulary corresponding to T1 is "exercise", and the synonym feature indicates that the vocabulary corresponding to T2 is "recipe". Taking "exercise" and "recipe" as extended vocabulary, the second keyword set obtained includes the following vocabulary: "fitness", "recipe", "exercise", and "recipe".

[0108] Step 240: extract features from the second keyword set to obtain keyword feature representation.

[0109] In some embodiments, the second keyword set may be further screened to improve the accuracy of the matching result, and features may be extracted from the search condition content to obtain a first sentence feature representation.

[0110] The first sentence feature representation and keyword feature representation are processed by a text sorting algorithm to obtain a semantic similarity image.

[0111] The semantic similarity image includes a main node representing the search condition content and multiple sub-nodes representing the vocabulary in the second keyword set. The multiple connections between the main node and the multiple sub-nodes correspond to node weights respectively, and the node weights are used to indicate the semantic similarity between the vocabulary in the second keyword set and the search condition content.

[0112] The node weights in the semantic similarity image are iteratively calculated and updated through the text sorting algorithm.

[0113] When the number of iterations reaches a preset iteration threshold, the second keyword set is screened based on the node weight to obtain a third keyword set; and feature extraction is performed on the third keyword set to obtain a keyword feature representation.

[0114] Optionally, the semantic similarity (node ​​weight) between each keyword in the second keyword set and the search condition content can be obtained through a text sorting algorithm. Based on the semantic similarity, the words in the second keyword set are sorted in descending order of similarity, and the words in the top 50% of the sequence are screened out as the words in the third keyword set.

[0115] For example, the second keyword set contains 8 words, namely word 1, word 2...word 8, and the semantic similarities between them and the search condition content are 0.81, 0.43, 0.55, 0.90, 0.91, 0.72, 0.77, and 0.98, respectively. The 8 words are sorted according to the similarity as follows: word 8 (0.98), word 5 (0.91), word 4 (0.90), word 1 (0.91), word 7 (0.77), word 6 (0.72), word 3 (0.55), word 2 (0.43), and the top 50% of the words (including word 8, word 5, word 4 and word 1) are determined as the third keyword set.

[0116] Exemplarily, the feature representation set corresponding to the second keyword set is V = {v1, v2, ..., vn}, where each vi is the d-dimensional feature representation of keyword i, the feature representation of the first sentence is S, and the merged feature representation set is C = {S, v1, v2, ..., vn}.

[0117] The above set C is used as input to the text ranking algorithm to construct a semantic similarity graph, where nodes represent keywords or search terms. Based on the semantic similarity or co-occurrence relationship between nodes, edges are constructed and assigned initial weights. The text ranking algorithm iteratively calculates and updates the weight of each node. This iterative calculation is repeated until the node weights converge or the preset number of iterations is reached. Finally, the keywords with the highest weights are selected to obtain the third keyword set.

[0118] For example, for two keyword feature representations vi and vj, the cosine similarity sim(vi,vj) between them is calculated. In the text sorting algorithm, the node weight Score(vi) is updated as follows:

[0119]

[0120] Where d is the damping coefficient, usually set to 0.85, In(vi) is the set of nodes pointing to node vi, and outDegree(vj) is the out-degree of node vj (the set of all nodes that can be reached from node vj). The above formula is iterated until the preset convergence condition is met or the preset number of iterations is reached.

[0121] Step 250 : Based on the matching degree between the keyword feature representation and the candidate content in the candidate content library, obtain at least one first content from the candidate content library as a search result of the search condition content.

[0122] Optionally, feature extraction is performed on the candidate content in the candidate content library to obtain a feature representation pool corresponding to the candidate content library. The candidate content belongs to sentence content. There is a mapping relationship between the nth sentence feature representation in the feature representation pool and the nth sentence content in the candidate content, where n is a positive integer.

[0123] Based on the similarity between the keyword feature representation and the sentence feature representation in the feature representation pool, a candidate content list is determined from the candidate content library, where the candidate content list includes a plurality of first contents.

[0124] Exemplarily, a pre-trained HNSW model (Hierarchical Navigable Small World, high-speed channel model) is used to calculate the keyword feature representation to determine the candidate content list.

[0125] The training process is as follows: A corpus containing multiple text segments, including but not limited to articles, news, and poetry, is obtained. The corpus is preprocessed, including removing special content (such as punctuation, numbers, and special characters), converting uppercase characters to lowercase, and removing stop words. The text is then split into single sentences [u1, u2, …, um]. Each sentence is vectorized using the BERT model to obtain a sentence vector representation [s1, s2, …, sm]. The vector representation [s1, s2, …, sm] is then fed into the HNSW model for training.

[0126] At the end of the training process, the corpus is processed to obtain a candidate content library, which contains multiple sentence contents. During the HNSW model training process, these sentence contents are mapped to form a feature representation pool.

[0127] Optionally, the keyword feature representation is input into a pre-trained HNSW model to calculate a candidate content list.

[0128] For example, for each feature representation si in the keyword feature representation, the HNSW model is used to search the feature representation pool for the set of sentence feature representations R(si) that are most similar to si. The cosine similarity between each sentence feature representation vj in R(si) and the keyword feature representation si is calculated. The search results are sorted according to the cosine similarity score, and the K most similar sentence feature representations with the highest ranking are returned. A candidate content list is then determined from the candidate content library based on the sentence content corresponding to the K sentence feature representations.

[0129] Optionally, based on the matching degree between the feature representation of the first sentence and the feature representations of the plurality of first contents, at least one first content is determined from the candidate content list as the search result of the search condition content.

[0130] Exemplarily, the degree of matching between the feature representation of the first sentence and the feature representations of the plurality of first contents is determined based on the cosine distance, the Pearson distance, and the BM25 correlation value.

[0131] Among them, cosine distance is a method to measure the difference between two vectors. The distance is calculated based on the cosine similarity of the vectors. Cosine similarity measures the degree of similarity between two vectors in direction. Pearson distance refers to the complement of the Pearson correlation coefficient and is used to measure the degree of correlation between two variables. The BM25 (Best Matching 25) algorithm is a relevance scoring algorithm based on the TF-IDF (Term Frequency-Inverse Document Frequency) model, and is improved to take into account factors such as document length. The BM25 algorithm evaluates the relevance between a document and a query by calculating the frequency (TF) and inverse document frequency (IDF) of the query terms in the document and adjusting it based on the document length.

[0132] Obtain the sentence feature representation of each first content, the first content feature representation set R1, where the i-th feature in R1 is represented as R_i, use the BERT model to encode the search condition content and multiple first contents respectively, and calculate the matching score match_score.

[0133] Calculate the cosine distance cosine, Pearson distance pearson and BM25 correlation value bm25 between S and each feature representation R_i in the R1 set.

[0134] The matching score match_score, cosine distance cosine, Pearson distance pearson and BM25 correlation value bm25 are used as features for the next step of fine ranking and input into the pre-trained fine ranking model LGBMRanker to obtain the ranking results after fine ranking. The top several first contents in the ranking results are returned to the user as search results.

[0135] In summary, the method provided by this application analyzes the search criteria input by the user, and for search criteria that belong to a sentence type, determines the amount of attention each word receives from the other words, thereby determining the importance of each word in the sentence, and screening keywords as the basis for expansion. By expanding the vocabulary of keywords in combination with the context, it is possible to more accurately understand the semantics of the search criteria content based on the expanded vocabulary, fully considering the possible ambiguity, puns, and other issues in the text content. Compared with the related art that relies solely on surface keywords for matching and then feeding back search results, it can improve the accuracy of the matching results and feedback results that better meet the search requirements.

[0136] In some embodiments, the solution provided by the present application can also expand the vocabulary search conditions entered by the user and return search results to the user based on the expanded vocabulary set. Figure 34 is a flowchart of a content search method provided by another exemplary embodiment of the present application, which includes the following steps.

[0137] Step 310: Obtain search condition content and determine the dimension of the search condition content.

[0138] Users can enter words, sentence fragments, sentences, etc. of any dimension as search condition content, and judge based on the length of the search condition. If the length is less than 5, it is judged as a word; if the length is greater than or equal to 5, it is judged as a sentence or sentence fragment. Hereinafter, sentences or sentence fragments are referred to as sentence content.

[0139] For example, if the search condition content is as follows: "health, diet, recipes", the length of the search condition content is based on the number of Chinese characters, and if it contains 6 Chinese characters, the search condition content is a sentence content.

[0140] For example, if the search condition content is as follows: "health, recipe", the length of the search condition content is based on the number of Chinese characters, and if it contains 4 Chinese characters, then the search condition content is vocabulary content.

[0141] Step 321 : When the length of the search condition content indicates that the search condition content belongs to sentence content, attention analysis is performed on the search condition content to determine a first keyword set of the search condition content.

[0142] The search condition content is segmented to obtain a sequence of words arranged in order after the segmentation, and each word in the sequence has a corresponding serial number.

[0143] The word sequence is processed through multiple transformer layers in the deep learning model (BERT model) and the self-attention weight matrix is ​​output.

[0144] The BERT model is used to extract features from the search condition content to obtain the feature representation of the first sentence. The self-attention weight matrix is ​​adjusted based on the feature representation of the first sentence. The weighted attention result obtained by each word from other words is calculated as the attention score and sorted by score. A preset number of words with higher scores are determined as the first keyword set.

[0145] Step 331 : perform vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary and a second keyword set including the first keyword set and the expanded vocabulary.

[0146] The first keyword set is mapped into a vector space, and an expanded vocabulary is determined based on distances between a plurality of vectors in the vector space and feature representations of the vocabulary in the first keyword set.

[0147] Step 341: perform feature extraction on the second keyword set to obtain keyword feature representation.

[0148] In some embodiments, the words in the second keyword set may be further screened based on a text sorting algorithm to obtain a third keyword set, and feature extraction may be performed on the third keyword set to obtain a keyword feature representation.

[0149] Optionally, the semantic similarity between each keyword in the second keyword set and the search condition content can be obtained through a text sorting algorithm. Based on the semantic similarity, the words in the second keyword set are sorted in descending order of similarity, and the words in the top 50% of the sequence are screened out as the words in the third keyword set.

[0150] For example, the second keyword set contains 8 words, namely word 1, word 2...word 8, and the semantic similarities between them and the search condition content are 0.81, 0.43, 0.55, 0.90, 0.91, 0.72, 0.77, and 0.98, respectively. The 8 words are sorted according to the similarity as follows: word 8 (0.98), word 5 (0.91), word 4 (0.90), word 1 (0.91), word 7 (0.77), word 6 (0.72), word 3 (0.55), word 2 (0.43), and the top 50% of the words (including word 8, word 5, word 4 and word 1) are determined as the third keyword set.

[0151] Step 351 : Based on the matching degree between the keyword feature representation and the candidate content in the candidate content library, at least one first content is obtained from the candidate content library as a search result of the search condition content.

[0152] Based on the keyword feature representation, a candidate content list is filtered out from the candidate content library. The candidate content list contains multiple first contents. The multiple first contents in the candidate content list are finely ranked based on the cosine distance, Pearson distance and BM25 correlation value. According to the fine ranking results, the first content with the highest ranking is used as the search result that needs to be finally returned to the user end.

[0153] Step 322: If the length of the search condition content indicates that the search condition content belongs to vocabulary content, the search condition content is modified to obtain a modified vocabulary.

[0154] First, homophone correction is performed on the search conditions to correct homophone typos in the search conditions and obtain corrected vocabulary.

[0155] For example, the search condition is the word "nutritious and good food". A pre-prepared word list is obtained, which contains a variety of common words. Homophones are found in the word list based on the pronunciation of "nutritious diet" to determine whether there are any typos.

[0156] After searching, it was found that "nutrition" existed in the vocabulary, but "good food" did not exist. Based on the pronunciation of "good food", the homonym "meal" was found, and the search conditions were modified to obtain the modified vocabulary "nutrition diet".

[0157] Step 332 , performing vocabulary expansion on the revised vocabulary to obtain an expanded vocabulary and an expanded vocabulary set including the expanded vocabulary and search condition content.

[0158] Keyword association is performed on the revised vocabulary to obtain an expanded vocabulary, and the expanded vocabulary is combined with the revised vocabulary to obtain an expanded vocabulary set.

[0159] The semantic similarity or identity between the expanded vocabulary and the revised vocabulary can be expanded in terms of feature representation and text semantics.

[0160] For example, the revised vocabulary is split into "nutrition" and "diet", and articles containing the keywords "nutrition" and "diet" are searched in the preset corpus to further determine whether the revised vocabulary appears in the same context at a frequency reaching a first preset threshold. Based on the corresponding articles in the context, other vocabulary with a higher frequency of appearance (the frequency of appearance reaches a second preset threshold) is searched as an expanded vocabulary.

[0161] For example, "nutrition" and "diet" frequently appear in the context of articles on health topics. Other common high-frequency words in this type of topic include "balance" and "vegetables". In this case, "balance" and "vegetables" can be used as extended words.

[0162] In some embodiments, the expanded vocabulary can also be determined by the distance between feature representations.

[0163] Optionally, feature extraction is performed on the revised vocabulary to obtain a revised word feature representation. A vocabulary and a feature representation library corresponding to the vocabulary are obtained. A mapping relationship exists between multiple word feature representations in the feature representation library and multiple words in the vocabulary. The k-th word feature representation in the feature representation library is obtained by feature extraction of the k-th word in the vocabulary, where k is a positive integer. A synonym feature representation corresponding to the revised word feature representation is determined in the feature representation library.

[0164] For the mth word feature representation in the revised word feature representation, the similarity between the mth word feature representation and multiple word feature representations in the feature representation library is calculated, and the word feature representation whose similarity reaches a preset threshold is determined as the synonym feature representation corresponding to the mth word feature representation. m is a positive integer.

[0165] Based on the synonym feature representation, corresponding synonyms are determined in the vocabulary as extended vocabulary, wherein the semantics of the synonyms and the semantics of the revised vocabulary meet a preset similarity requirement.

[0166] Exemplarily, the pre-trained Word2Vec model is used to extract features of the corrected vocabulary to obtain a corrected word feature representation. Since the number of corrected word feature representations is at least one, the corrected word feature representation set is represented as [w1, w2, ..., wm]. For any vocabulary in the set, the Word2Vec model maps it to a d-dimensional feature representation.

[0167] Based on the similarity calculation function of the Word2Vec model, the feature representation with the highest similarity to each feature representation in the modified word feature representation set is determined from the feature representation library as the synonym feature representation.

[0168] Based on the vocabulary corresponding to the synonym feature representation, the synonyms of the revised vocabulary are determined, and the synonyms and the revised vocabulary are used as an extended vocabulary set.

[0169] It is worth noting that since the semantics of each revised vocabulary may be similar, it is necessary to filter the synonyms obtained based on the synonym feature representation to avoid synonym duplication.

[0170] Step 342: extract features from the expanded vocabulary set to obtain expanded word feature representations.

[0171] In some embodiments, the expanded vocabulary set may be further screened to improve the accuracy of the matching results.

[0172] Feature extraction is performed on all words in the extended vocabulary set to obtain extended word feature representations corresponding to the extended vocabulary and search word feature representations corresponding to the search condition content.

[0173] The search term feature representation and the extended term feature representation are processed by a text sorting algorithm to obtain a semantic similarity image.

[0174] The semantic similarity image includes a main node representing the search condition content and multiple sub-nodes representing extended vocabulary. The multiple connections between the main node and the multiple sub-nodes correspond to node weights, which are used to indicate the semantic similarity between the extended vocabulary and the search condition content.

[0175] The node weights in the semantic similarity image are iteratively calculated and updated through the text sorting algorithm.

[0176] When the number of iterations reaches a preset iteration threshold, the expanded vocabulary set is screened based on the node weights to obtain a matching vocabulary set.

[0177] Feature extraction is performed on the matching vocabulary set to obtain matching word feature representations, which can be used to match candidate content in the candidate content library to determine search results to be fed back to the user.

[0178] Step 352: Based on the matching degree between the expanded word feature representation and the candidate content in the candidate content library, obtain at least one second content from the candidate content library as a search result of the search condition content.

[0179] A candidate content list is screened out from the candidate content library based on the extended word feature representation (or the matching word feature representation), where the candidate content list includes a plurality of second contents, and the second contents may be expressed in the form of words, sentences, or sentence fragments.

[0180] The multiple second-person contents in the candidate content list are finely ranked based on the cosine distance, Pearson distance and BM25 correlation value, and the second-person contents with the highest ranking are used as the search results that need to be finally returned to the user end according to the fine ranking result.

[0181] It is worth noting that the contents executed from step 321 to step 351 are in parallel with the contents executed from step 322 to step 352, that is, the contents from step 321 to step 351 may be executed first, and then the contents from step 322 to step 352; or the contents from step 322 to step 352 may be executed first, and then the contents from step 321 to step 351; or the contents from step 321 to step 351 and the contents from step 322 to step 352 may be executed at the same time.

[0182] To sum up, the content search method provided by this application can simultaneously meet the needs of result feedback for multi-dimensional search conditions, analyze the vocabulary in the search condition content in combination with the context, fully understand the possible puns, ambiguities and other phenomena in the text content, and expand the search condition content to obtain search results with higher matching degrees, thereby improving the content search feedback effect.

[0183] Figure 4 This is a structural block diagram of a content search device provided by an exemplary embodiment of the present application. Figure 4 As shown, the device includes the following parts.

[0184] An acquisition module 410 is used to acquire search condition content, where the search condition content is content input by a user for searching;

[0185] an attention analysis module 420 configured to, when the length of the search condition content indicates that the search condition content is sentence content, perform attention analysis on the search condition content to determine a first keyword set for the search condition content, the first keyword set including at least one keyword in the search condition content, the at least one keyword being a word in the search condition content that meets a preset attention condition;

[0186] a vocabulary expansion module 430 configured to perform vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary, and a second keyword set comprising the first keyword set and the expanded vocabulary, wherein the expanded vocabulary meets a preset correlation requirement with the keywords in the first keyword set;

[0187] A feature extraction module 440 is configured to extract features from the second keyword set to obtain a keyword feature representation;

[0188] The search result determination module 450 is configured to obtain at least one first content from the candidate content library as a search result of the search condition content based on a matching degree between the keyword feature representation and the candidate content in the candidate content library.

[0189] In an optional embodiment, the attention analysis module 420 is further used to perform word segmentation processing on the search condition content to obtain a word sequence arranged in order after the segmentation, and each word in the word sequence corresponds to a serial number; the word sequence is processed by multiple transformer layers in the deep learning model, and a self-attention weight matrix is ​​output, wherein the i-th row element in the self-attention weight matrix is ​​used to indicate the attention distribution of the word with serial number i, and the j-th column element in the self-attention weight matrix is ​​used to indicate the attention weight assigned to each word in the word sequence to the word with serial number j, and the i-th row and j-th column in the self-attention weight matrix are The element represents the attention weight assigned by the word with sequence number i to the word with sequence number j in the word sequence; i and j are positive integers; the self-attention weight matrix is ​​adjusted to obtain a weighted self-attention weight matrix; the attention score of each word in the word sequence is determined based on the weighted self-attention weight matrix, and the attention score is used to indicate the importance of each word in the word sequence in the search condition content; the first keyword set of the search condition content is determined based on the attention score of each word in the word sequence, wherein the attention score of the words in the first keyword set is greater than a preset score threshold.

[0190] In an optional embodiment, the attention analysis module 420 is further used to perform feature extraction on the search condition content to obtain a first sentence feature representation; and determine a weighted self-attention weight matrix based on the product between the first sentence feature representation and the self-attention weight matrix.

[0191] In an optional embodiment, the vocabulary expansion module 430 is further used to perform feature extraction on the at least one keyword in the first keyword set to obtain at least one corresponding word feature representation; obtain a vocabulary and a feature representation library corresponding to the vocabulary, and there is a mapping relationship between the multiple word feature representations in the feature representation library and the multiple words in the vocabulary, wherein the k-th word feature representation in the feature representation library is obtained after feature extraction of the k-th word in the vocabulary, and k is a positive integer; determine the synonym feature representation corresponding to the at least one word feature representation in the feature representation library; determine the corresponding synonym in the vocabulary as the extended vocabulary based on the synonym feature representation, wherein the semantics of the synonym and the semantics of the at least one keyword meet the preset similarity requirements.

[0192] In an optional embodiment, the vocabulary expansion module 430 is further used to calculate the similarity between the mth word feature representation in the at least one word feature representation and the multiple word feature representations in the feature representation library, and determine the word feature representation whose similarity reaches a preset threshold as the synonym feature representation corresponding to the mth word feature representation; m is a positive integer.

[0193] In an optional embodiment, the feature extraction module 440 is further configured to extract features from the search condition content to obtain a first sentence feature representation;

[0194] like Figure 5 As shown, the device also includes:

[0195] The vocabulary screening module 460 is configured to process the first sentence feature representation and the keyword feature representation using a text sorting algorithm to obtain a semantic similarity image, wherein the semantic similarity image includes a main node representing the search condition content and multiple sub-nodes representing the vocabulary in the second keyword set, and multiple links between the main node and the multiple sub-nodes respectively correspond to node weights, and the node weights are used to indicate the semantic similarity between the vocabulary in the second keyword set and the search condition content; iteratively calculate and update the node weights in the semantic similarity image using the text sorting algorithm; and when the number of iterations reaches a preset iteration number threshold, screen the second keyword set based on the node weights to obtain a third keyword set;

[0196] The feature extraction module 440 is further configured to perform feature extraction on the third keyword set to obtain the keyword feature representation.

[0197] In an optional embodiment, the search result determination module 450 is further used to perform feature extraction on the candidate content in the candidate content library to obtain a feature representation pool corresponding to the candidate content library, the candidate content belongs to sentence content, and there is a mapping relationship between the nth sentence feature representation in the feature representation pool and the nth sentence content in the candidate content, where n is a positive integer; based on the similarity between the keyword feature representation and the sentence feature representation in the feature representation pool, a candidate content list is determined from the candidate content library, and the candidate content list includes multiple first contents; based on the matching degree between the first sentence feature representation and the feature representations of the multiple first contents, the at least one first content is determined from the candidate content list as the search result of the search condition content.

[0198] In summary, the content search device provided by the present application can analyze the search condition content input by the user, and for the search condition content belonging to the sentence type, determine the amount of attention each word receives from the other words, so as to determine the importance of each word in the sentence, and select keywords as the basis for expansion. By expanding the vocabulary of keywords in combination with the context, it is possible to more accurately understand the semantics of the search condition content based on the expanded vocabulary, and fully consider the possible ambiguity, puns and other problems in the text content. Compared with the related art that relies only on surface keywords for matching and then feedback search results, it can improve the accuracy of the matching results and feedback results that better meet the search requirements.

[0199] It should be noted that the content search device provided in the above embodiment is merely an example of the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the content search device provided in the above embodiment and the content search method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0200] Figure 6The following is a block diagram of a computer device 600 according to an exemplary embodiment of the present application. The computer device 600 may be a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The computer device 600 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other similar names.

[0201] Typically, the computer device 600 includes a processor 601 and a memory 602 .

[0202] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0203] Memory 602 may include one or more computer-readable storage media, which may be non-transitory. Memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 602 is used to store at least one instruction, which is executed by processor 601 to implement the content search method provided in the method embodiment of the present application.

[0204] In some embodiments, the computer device 600 further includes some other components 603, and the type and quantity of the other components 603 can be selected based on the functional requirements of the computer device 600. It will be understood by those skilled in the art that Figure 6 The structure shown in the figure does not constitute a limitation on the computer device 600, and the computer device 600 may include more or fewer components than shown in the figure, or combine some components, or adopt a different arrangement of components.

[0205] Optionally, the computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid-state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0206] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the content search method as described in any of the above embodiments of the present application.

[0207] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the content search method as described in any of the above embodiments of the present application.

[0208] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the content search method described in any of the above embodiments.

[0209] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0210] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A content search method, characterized in that: The method comprises: Acquire search condition content, where the search condition content is content input by a user for searching; When the length of the search condition content indicates that the search condition content is a sentence content, performing attention analysis on the search condition content to determine a first keyword set for the search condition content, the first keyword set including at least one keyword in the search condition content, the at least one keyword being a word in the search condition content that meets a preset attention condition; performing vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary, and a second keyword set including the first keyword set and the expanded vocabulary, wherein the expanded vocabulary meets a preset correlation requirement with the keywords in the first keyword set; Extracting features from the search condition content to obtain a first sentence feature representation; Processing the first sentence feature representation and the keyword feature representation using a text sorting algorithm to obtain a semantic similarity image, wherein the semantic similarity image includes a main node representing the search condition content and multiple sub-nodes representing words in the second keyword set, and multiple links between the main node and the multiple sub-nodes are respectively associated with node weights, and the node weights are used to indicate semantic similarities between the words in the second keyword set and the search condition content; Iteratively calculating and updating the node weights in the semantic similarity image using the text sorting algorithm; When the number of iterations reaches a preset iteration number threshold, filtering the second keyword set based on the node weight to obtain a third keyword set; performing feature extraction on the third keyword set to obtain the keyword feature representation; Based on the matching degree between the keyword feature representation and the candidate content in the candidate content library, at least one first content is obtained from the candidate content library as a search result of the search condition content.

2. The method according to claim 1, characterized in that The performing attention analysis on the search condition content to determine a first keyword set of the search condition content includes: Performing word segmentation on the search condition content to obtain a word sequence arranged in order after the segmentation, wherein each word in the word sequence corresponds to a serial number; The word sequence is processed by multiple transformer layers in a deep learning model to output a self-attention weight matrix, wherein the i-th row element in the self-attention weight matrix is ​​used to indicate the attention distribution of the word with sequence number i, the j-th column element in the self-attention weight matrix is ​​used to indicate the attention weight assigned by each word in the word sequence to the word with sequence number j, and the i-th row and j-th column element in the self-attention weight matrix represents the attention weight assigned by the word with sequence number i to the word with sequence number j in the word sequence; i and j are positive integers; Adjusting the self-attention weight matrix to obtain a weighted self-attention weight matrix; determining an attention score for each word in the word sequence based on the weighted self-attention weight matrix, wherein the attention score is used to indicate the importance of each word in the word sequence in the search condition content; The first keyword set of the search condition content is determined based on the attention score of each word in the word sequence, wherein the attention score of the words in the first keyword set is greater than a preset score threshold.

3. The method according to claim 2, characterized in that The self-attention weight matrix is ​​adjusted to obtain a weighted self-attention weight matrix, including: Extracting features from the search condition content to obtain a first sentence feature representation; A weighted self-attention weight matrix is ​​determined based on the product between the first sentence feature representation and the self-attention weight matrix.

4. The method according to claim 1, wherein The step of performing vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary includes: performing feature extraction on the at least one keyword in the first keyword set to obtain at least one corresponding word feature representation; Obtain a vocabulary and a feature representation library corresponding to the vocabulary, wherein a mapping relationship exists between multiple word feature representations in the feature representation library and multiple words in the vocabulary, wherein the k-th word feature representation in the feature representation library is obtained after feature extraction of the k-th word in the vocabulary, where k is a positive integer; determining, in the feature representation library, a synonym feature representation corresponding to the at least one word feature representation; Based on the synonym feature representation, a corresponding synonym is determined in the vocabulary as the extended vocabulary, wherein the semantics of the synonym and the semantics of the at least one keyword meet a preset similarity requirement.

5. The method according to claim 4, characterized in that The determining, in the feature representation library, a synonym feature representation corresponding to the at least one word feature representation comprises: For the mth word feature representation in the at least one word feature representation, calculate the similarity between the mth word feature representation and the multiple word feature representations in the feature representation library, and determine the word feature representation whose similarity reaches a preset threshold as the synonym feature representation corresponding to the mth word feature representation; m is a positive integer.

6. The method according to any one of claims 1 to 5, characterized in that: The acquiring at least one first content from the candidate content library as a search result of the search condition content based on the matching degree between the keyword feature representation and the candidate content in the candidate content library includes: Performing feature extraction on candidate content in the candidate content library to obtain a feature representation pool corresponding to the candidate content library, wherein the candidate content is sentence content, and an nth sentence feature representation in the feature representation pool is mapped to an nth sentence content in the candidate content, where n is a positive integer; determining a candidate content list from the candidate content library based on a similarity between the keyword feature representation and a sentence feature representation in the feature representation pool, wherein the candidate content list includes a plurality of first contents; Based on the matching degree between the first sentence feature representation and the feature representations of the plurality of first contents, the at least one first content is determined from the candidate content list as the search result of the search condition content.

7. A content search device, characterized in that: The device comprises: An acquisition module is used to acquire search condition content, where the search condition content is content input by a user for searching; an attention analysis module, configured to, when the length of the search condition content indicates that the search condition content is sentence content, perform attention analysis on the search condition content to determine a first keyword set for the search condition content, the first keyword set including at least one keyword in the search condition content, the at least one keyword being a word in the search condition content that meets a preset attention condition; a vocabulary expansion module configured to perform vocabulary expansion on at least one keyword in the first keyword set to obtain an expanded vocabulary, and a second keyword set comprising the first keyword set and the expanded vocabulary, wherein the expanded vocabulary meets a preset correlation requirement with the keywords in the first keyword set; A feature extraction module is configured to perform feature extraction on the search condition content to obtain a first sentence feature representation; process the first sentence feature representation and the keyword feature representation through a text sorting algorithm to obtain a semantic similarity image, wherein the semantic similarity image includes a main node representing the search condition content and multiple sub-nodes representing words in the second keyword set, and multiple links between the main node and the multiple sub-nodes respectively correspond to node weights, and the node weights are used to indicate the semantic similarity between the words in the second keyword set and the search condition content; iteratively calculate and update the node weights in the semantic similarity image through the text sorting algorithm; when the number of iterations reaches a preset iteration number threshold, screen the second keyword set based on the node weights to obtain a third keyword set; perform feature extraction on the third keyword set to obtain the keyword feature representation; The search result determination module is configured to obtain at least one first content from the candidate content library as a search result of the search condition content based on a matching degree between the keyword feature representation and the candidate content in the candidate content library.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the content search method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the content search method according to any one of claims 1 to 6.

Images