Text matching method and device, and electronic device
Patent Information
- Application Number
- CN202211211508.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-30
- Publication Date
- 2026-09-04
- Estimated Expiration
- 2042-09-30
AI Technical Summary
[0005]本申请提供一种文本匹配方法、装置及电子设备,用以解决现有的问答场景中查询获得的目标文本与查询文本不匹配的技术问题
[0020]本申请提供的技术方案中,通过响应于查询文本的查询请求,从问答库中查询满足相似条件的候选文本集,以基于查询文本的字符串长度和候选文本集中候选文本的字符串长度,确定每个候选文本的第一阈值,该第一阈值是根据查询文本和与其候选文本的字符串长度实时确定的,使得第一阈值与对应的候选文本关联更紧密。通过每个候选文本的第一阈值可以对该候选文本是否满足阈值比较条件进行判断,实现每个候选文本的独立化判断,提高候选文本的阈值比较条件的准确度,使得目标候选文本集的选择与候选文本自身的关联更紧密,准确度更高,因此,再从目标候选文本中确定与查询文本相似度最高的目标文本,可以实现对目标文本的更详细的选择,提高目标文本的选择效率和精度。
Smart Images

Figure CN116127005B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to a text matching method, apparatus and electronic device. Background Technology
[0002] FAQ (Frequently Asked Questions) dialogue systems are the most common method used in intelligent customer service. A dialogue system can be configured with an FAQ database, which stores question-answer pairs associated with "standard questions and standard answers." Related technologies allow for the expansion of standard questions into multiple similar questions. The question-answer database formed by the standard question and its corresponding similar questions can then be used for semantic matching.
[0003] In related technologies, users can input query text. When searching for similar questions for the query text, the process generally involves two steps: question recall and similarity ranking. The question recall step primarily uses a coarse-grained matching method to search for candidate texts similar to the query text from a question-and-answer database consisting of standard questions and their corresponding similar questions. The similarity ranking step mainly calculates the information similarity between the recalled candidate texts and the query text, determining the candidate text with the highest similarity as the target text.
[0004] However, for query texts with a large number of words, the accuracy of the results obtained through the existing two steps of question recall and similarity ranking is not high. Summary of the Invention
[0005] This application provides a text matching method, apparatus, and electronic device to solve the technical problem of mismatch between the target text obtained from the query and the query text in existing question-and-answer scenarios.
[0006] Firstly, this application provides a text matching method, including:
[0007] In response to a query request for a query text, a set of candidate texts that meet similar conditions is retrieved from a question-and-answer database, wherein the set of candidate texts includes at least one candidate text.
[0008] Based on the string length of the query text and the string length of each candidate text in the candidate text set, a first threshold is determined for each candidate text. The first threshold is used to measure the similarity between the candidate text and the query text.
[0009] Using the first threshold corresponding to each candidate text, a subset of target candidate texts that meet the threshold comparison conditions is determined from the candidate text set;
[0010] The target text with the highest similarity to the query text is determined from the subset of target candidate texts.
[0011] Secondly, this application provides a text matching device, comprising:
[0012] The candidate determination unit is used to query candidate texts that meet similar conditions from the question-and-answer database in response to a query request for query text;
[0013] A threshold determination unit is used to determine a first threshold corresponding to the candidate text based on the string lengths of the query text and the candidate text, wherein the first threshold is used to measure the similarity of the candidate text.
[0014] A threshold comparison unit is used to determine target candidate texts that meet the threshold comparison conditions from the candidate texts using a first threshold corresponding to the candidate texts;
[0015] The target determination unit is used to determine the target text with the highest similarity to the query text from the target candidate texts.
[0016] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;
[0017] The memory stores computer-executed instructions;
[0018] The processor executes computer execution instructions stored in the memory to implement the first aspect and various possible text matching methods of the first aspect.
[0019] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and various possible text matching methods of the first aspect.
[0020] In the technical solution provided in this application, in response to a query request for query text, a set of candidate texts that meet similarity conditions is retrieved from a question-and-answer database. A first threshold is determined for each candidate text based on the string length of the query text and the string lengths of the candidate texts in the candidate text set. This first threshold is determined in real-time based on the string lengths of the query text and its candidate texts, making the first threshold more closely associated with the corresponding candidate text. By using the first threshold for each candidate text, it is possible to determine whether the candidate text meets the threshold comparison conditions, achieving independent judgment for each candidate text and improving the accuracy of the threshold comparison conditions for candidate texts. This makes the selection of the target candidate text set more closely associated with the candidate texts themselves, resulting in higher accuracy. Therefore, further determining the target text with the highest similarity to the query text from the target candidate text allows for a more detailed selection of target texts, improving the efficiency and accuracy of target text selection. Attached Figure Description
[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0022] Figure 1 An application example diagram of a dialogue system provided in this application embodiment;
[0023] Figure 2 A flowchart illustrating one embodiment of a text matching method provided in this application;
[0024] Figure 3 A flowchart illustrating yet another embodiment of a text matching method provided in this application;
[0025] Figure 4 A flowchart illustrating yet another embodiment of a text matching method provided in this application;
[0026] Figure 5 A flowchart illustrating yet another embodiment of a text matching method provided in this application;
[0027] Figure 6 A schematic diagram of one embodiment of a text matching device provided in this application;
[0028] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.
[0029] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0031] The technical solution of this application can be applied to automatic question answering scenarios. After obtaining candidate texts, the threshold of the candidate texts is dynamically obtained, which improves the accuracy of threshold judgment of candidate texts. At the same time, after obtaining the screening results of the candidate thresholds, the target text with the highest similarity is selected, which increases the accuracy of the target text for querying the query text.
[0032] In related technologies, dialogue systems typically employ automatic question-and-answer methods. This involves maintaining a FAQ (FAQ) database, which stores question-and-answer pairs. When a user-input query is detected, the answer can be retrieved in two steps. The first step is generally coarse recall, which involves searching the database for candidate texts similar to the query. The second step is fine ranking, which calculates the similarity between the candidate texts and the query text to determine the most similar target text. In the fine ranking stage, the similarity calculation between the candidate texts and the query text can typically use a text matching model for similarity scoring. Text matching models can include deep learning models and pre-trained models such as LSTM and BERT, but the accuracy of similarity matching results obtained through text matching models is not high.
[0033] Research revealed that in the FAQ database, each question is a relatively short string, such as common questions like "Air conditioner won't start," "Air conditioner isn't cooling," "Shoes don't fit," and "Clothing size is incorrect." However, the length of the query text provided by users is much longer than the question string. For example, suppose the query text is "I bought a pair of shoes from you last week, and they arrived yesterday. I tried them on, and they feel a bit small. I always wear this size. Is there a problem with your sizing? How can I fix this?" The most suitable target text could actually be "Shoes don't fit." Because the query text is longer and contains a lot of redundant information, using conventional methods would result in a lower similarity between the query text and the question "Shoes don't fit," leading to the query text not matching the most accurate question.
[0034] To address the aforementioned technical issues, in this embodiment, based on the preliminary coarse recall to obtain candidate texts, a first threshold corresponding to the candidate text is determined by utilizing the string lengths of the query text and the candidate text. That is, the similarity measurement threshold of the candidate text is determined in real time by utilizing the string lengths of the query text and the candidate text. The threshold specifically set for the candidate text is used to perform more detailed screening of the candidate text itself, thereby improving the accuracy of the target text.
[0035] The problem query method provided in this application is intended to solve the above-mentioned technical problems in the prior art.
[0036] In the technical solution of this application, in response to a query request for query text, candidate texts that meet similarity conditions can be queried from a question-and-answer database. Based on the string lengths of the query text and the candidate texts, a first threshold corresponding to the candidate text is determined. This first threshold is determined in real time based on the string lengths of the query text and the candidate texts, and may be related to the string length of the candidate text itself. It can be used to measure the similarity of the corresponding candidate texts. Furthermore, the first threshold, which is more closely related to the candidate text, can be used to judge whether the candidate text meets the threshold comparison conditions, thereby obtaining the target candidate text that meets the threshold comparison conditions. The selection of the target candidate text is more closely related to the first threshold determined for the candidate text in real time, and the accuracy is higher. Therefore, by further determining the target text with the highest similarity to the query text from the target candidate texts, the target text can be selected more accurately, and the accuracy of the target text can be improved.
[0037] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0038] like Figure 1 The diagram shown is an application example of a dialogue system provided in an embodiment of this application. (Reference) Figure 1 The dialogue system may include server 1 and multiple electronic devices 2, assuming the multiple electronic devices include electronic devices 21, 22, 23, and 24 in the diagram. Server 1 may maintain a FAQ database 11, which may specifically store question-answer pairs 12 in the form of "question-answer". In some possible designs, the questions can be expanded to include multiple similar questions, and then "multiple similar question-answer pairs" can also constitute question-answer pairs in the database.
[0039] Any electronic device, such as one or more of electronic devices 21, 22, 23, and 24, can initiate a query request to server 1. The query request may include query text. Server 1 can query a set of candidate texts that meet similarity conditions based on question-answering database 11, and use a first threshold for each candidate text to measure the similarity of the candidate texts. The first threshold is determined by using the string length of the query text and the string length of each candidate text in the candidate text set. By using the first threshold, the similarity between the corresponding candidate text and the query text can be measured. Target candidate texts that meet the threshold comparison conditions can be selected from the candidate text set by comparing the first threshold of each candidate text. By determining the first threshold for each candidate text, real-time threshold selection judgment of candidate texts can be completed, obtaining more effective target candidate texts, and identifying the target text with the highest similarity to the query text from the target candidate texts.
[0040] In this embodiment, server 1 can be a regular server or a cloud server. The specific type of server is not limited in this application.
[0041] The aforementioned electronic devices can be either wireless or wired terminals. A wireless terminal can be a device that provides voice and / or other service data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core network devices via a Radio Access Network (RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or "cellular" phone) or a computer with a mobile terminal, for example, a portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile device, which exchanges voice and / or data with the RAN. Furthermore, a wireless terminal can also be a Personal Communication Service (PCS) phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA), or other similar devices. Wireless terminals can also be referred to as systems, subscriber units, subscriber stations, mobile stations, mobile stations, remote stations, remote terminals, access terminals, user terminals, user agents, user devices, or user equipment, without limitation. Optionally, the aforementioned electronic devices can also be smartwatches, tablets, etc.
[0042] like Figure 2 The diagram shown is a flowchart of one embodiment of a text matching method provided in this application. This text matching method can be configured as a text matching device, which can be located in an electronic device or a server. The text matching method may include the following steps:
[0043] 201: In response to a query request for a query text, retrieve a set of candidate texts from the question-and-answer database that meet similar criteria. The set of candidate texts includes at least one candidate text.
[0044] Optionally, the question-and-answer database may include multiple question-and-answer pairs. Any question-and-answer pair may include a question and the answer associated with that question. The question in a question-and-answer pair may include a base question and similar questions that expand upon that base question.
[0045] Querying candidate texts that meet similar conditions from a question-and-answer database can include: using the Elastic Search (ES) engine, querying the question-and-answer database for at least one candidate text that meets the similar conditions, in order to obtain a candidate text set consisting of at least one candidate text.
[0046] The technical solution of this application can be applied to electronic devices or servers, and this application does not impose too many restrictions on the subject executing the text matching method. Figure 1 The dialogue system shown is merely an example and should not be construed as a specific limitation on the application scenarios of the text matching method.
[0047] 202: Based on the string length of the query text and the string length of each candidate text in the candidate text set, determine the first threshold corresponding to each candidate text. The first threshold is used to measure the similarity between the corresponding candidate text and the query text.
[0048] The length of the query text can refer to the number of characters in the query text. The length of the candidate text can refer to the number of characters in the candidate text.
[0049] 203: Using the first threshold corresponding to each candidate text, determine the target candidate text subset that meets the threshold comparison condition from the candidate text set.
[0050] Each candidate text can be evaluated against a threshold comparison condition to filter it. If a candidate text meets the threshold comparison condition, it is determined to be a target candidate text; if it does not meet the threshold comparison condition, it is determined that it does not meet the usage requirements and can be ignored or deleted. The target candidate text subset can include at least one target candidate text that meets the threshold comparison condition. The threshold comparison condition allows for preliminary filtering of candidate texts, enabling an initial judgment on whether the threshold comparison condition meets the usage requirements.
[0051] For the threshold comparison process of each candidate text, the first threshold of the candidate text can be used to participate in the threshold comparison, so that the first threshold of each candidate text changes with the candidate text. The threshold comparison conditions of each candidate text are dynamically updated to improve the accuracy of threshold comparison.
[0052] 204: Identify the target text with the highest similarity to the query text from the subset of target candidate texts.
[0053] Optionally, determining the target text with the highest similarity to the query text from the subset of target candidate texts may include: determining the target text with the highest similarity to the query text from at least one target candidate text contained in the subset of target candidate texts. Target candidate texts may correspond to target similarity scores. Determining the target text with the highest similarity to the query text from at least one target candidate text may include: sorting the target similarity scores corresponding to each of the at least one target candidate text to obtain at least one sorted target candidate text, and determining the target text with the highest target similarity from the at least one sorted target candidate text.
[0054] In this application's technical solution, in response to a query request for query text, a set of candidate texts that meet similarity conditions is retrieved from a question-and-answer database. A first threshold is determined for each candidate text based on the string length of the query text and the string lengths of the candidate texts in the candidate text set. This first threshold is determined in real-time based on the string lengths of the query text and its candidate texts, making the first threshold more closely associated with the corresponding candidate text. By using the first threshold for each candidate text, it is possible to determine whether the candidate text meets the threshold comparison conditions, achieving independent judgment for each candidate text and improving the accuracy of the threshold comparison conditions for candidate texts. This makes the selection of the target candidate text set more closely associated with the candidate texts themselves, resulting in higher accuracy. Therefore, further determining the target text with the highest similarity to the query text from the target candidate text allows for a more detailed selection of target texts, improving the efficiency and accuracy of target text selection.
[0055] Furthermore, the technical solution disclosed herein can achieve dual filtering by dynamically comparing the similarity between candidate text and query text using thresholds and selecting the segment with the highest threshold. Compared to related technologies that directly use fixed threshold comparisons between candidate text and query text, this solution offers faster speed and higher recognition accuracy. Simultaneously, this solution does not require excessive expert-annotated data to save labor costs, can expand the question-and-answer database to increase the filtering range, and adapts to character matching of varying lengths, resulting in greater recognition accuracy and versatility.
[0056] As an example, such as Figure 3 The diagram shown is a flowchart of one embodiment of a text matching method provided in this application. This text matching method can be configured as a text matching device, which can be located in an electronic device. Figure 2 The difference in the illustrated embodiment lies in the method for determining the first threshold corresponding to each candidate text, which includes:
[0057] 301: Based on the length of the first string of the query text and the length of the second string of the corresponding candidate text, obtain the string ratio of the length of the first string to the length of the second string;
[0058] 302: Determine the first threshold of candidate text based on the string ratio and the preset target value.
[0059] Optionally, the length of the first string can be the number of characters in the query text, and can be represented by the letter 'l'. The length of the second string can be the number of characters in the candidate text, and can be represented by the letter 'y'. The string ratio can be calculated by dividing the length of the first string by the length of the second string. The first threshold can be obtained by numerically calculating the string ratio and a preset target threshold.
[0060] In this embodiment, the string ratio of the query text and the candidate text can be obtained by comparing the first string length of the query text and the second string length of the candidate text. Using this string ratio and a preset target value, a first threshold for the candidate text can be accurately determined. The string ratio is related to the string length of the candidate text itself, achieving the goal of setting a corresponding first threshold for each candidate text. This first threshold is then used to perform real-time threshold comparison conditions for the candidate text, improving the comparison effect.
[0061] In one possible design, determining a first threshold for candidate text based on the string ratio and a preset target value can include:
[0062] If the string ratio is greater than or equal to the target value, the first threshold of the candidate text is determined by multiplying the preset initial threshold and the preset first threshold coefficient.
[0063] If the string ratio is less than the target value, then the first threshold of the candidate text is determined based on the initial threshold and the string ratio.
[0064] Optionally, the target value can be set according to usage requirements. For example, the target value can be set to 4. The initial threshold and the first threshold coefficient can be preset. Let β represent the initial threshold, and assume the threshold coefficient is: The first threshold can then be expressed as:
[0065] Where l ≥ 4y. l can be the length of the first string, and y can be the length of the second string.
[0066] Of course, in practical applications, the target value, initial threshold, and first threshold system can all be set according to actual usage requirements. The first threshold can be calculated based on the ratio of the initial threshold to the string value.
[0067] In this embodiment, the string length relationship, i.e., the string ratio, between the query text and the candidate text is obtained by utilizing the first string length of the query text and the second string length of the candidate text, thus achieving accurate acquisition of the string length relationship between the query text and the candidate text. By comparing the string ratio with the target value, a first threshold for longer candidate texts can be obtained by multiplying an initial threshold by a first threshold coefficient. For shorter candidate texts, the initial threshold and string ratio can be used to specifically determine the candidate texts. Through the classification calculation of the first threshold, accurate and real-time first thresholds can be obtained for candidate texts of different lengths, allowing for early screening of candidate texts and ensuring that the obtained target candidate texts meet the corresponding similarity measurement results, resulting in more accurate target candidate texts.
[0068] As one example, determining a first threshold for candidate text based on an initial threshold and a string ratio may include:
[0069] The second threshold coefficient is calculated based on the string ratio; the second threshold coefficient is used to indicate the similarity comparison threshold related to the string length of the candidate text.
[0070] Calculate the product of the second threshold coefficient and the initial threshold to obtain the first threshold of the candidate text.
[0071] In this embodiment, the string ratio can be used to determine a second threshold coefficient, which is then multiplied by an initial threshold to obtain a first threshold for the candidate text. Using the string ratio to determine the second threshold coefficient allows for combining the string ratio of the candidate text and the query text with the second threshold coefficient of the candidate text, resulting in a higher degree of adaptability between the second threshold coefficient and the string length of the candidate text, and a more accurate first threshold.
[0072] The calculation of the second threshold coefficient based on the string ratio can include:
[0073] Calculate the product of the string ratio and pi to obtain the first ratio data;
[0074] Calculate the quotient of the first ratio data and the preset first constant to obtain the second ratio data;
[0075] Calculate the cosine value of the second ratio data to obtain the second threshold coefficient.
[0076] Optionally, the string ratio is: The first constant can be set to 16. The second ratio data can be expressed as:
[0077] The formula for calculating the second threshold coefficient can be expressed as: Where l < 4y.
[0078] l can be the length of the first string, and y can be the length of the second string.
[0079] In this embodiment of the application, the string length correlation between candidate text and query text can be accurately extracted by using string ratio, pi, and a first constant, and an accurate second threshold coefficient can be obtained by calculation.
[0080] As an example, such as Figure 4 The diagram shown is a flowchart of one embodiment of a text matching method provided in this application. This text matching method can be configured as a text matching device, which can be located in an electronic device. Figure 2 The difference in the illustrated embodiment is that the candidate text includes at least one, each candidate text includes a target similarity and a first threshold, and the method further includes:
[0081] 401: Calculate the similarity between each candidate text and the query text to obtain the target similarity for each candidate text.
[0082] Target similarity can be determined based on the degree of word segmentation similarity between the word segmentation of the candidate text and the word segmentation of the query text.
[0083] Optionally, calculating the target similarity between each candidate text and the query text may include: calculating the number of words in each candidate text that are the same as the words in the query text, thus obtaining the number of the same words corresponding to each candidate text; and calculating the ratio of the number of the same words corresponding to each candidate text to the total number of words corresponding to all word segments, thus obtaining the target similarity of each candidate text.
[0084] Step 203: Using the first threshold corresponding to each candidate text, determine the target candidate text subset that meets the threshold comparison condition from the candidate text set, which may include:
[0085] 402: Based on the target similarity and the first threshold corresponding to each candidate text, determine the target candidate text that meets the threshold comparison condition from the candidate text set.
[0086] 403: Based on the target candidate text, determine a subset of the target candidate text.
[0087] Threshold comparison conditions can refer to a strategy for determining whether a candidate text meets the usage conditions using a first threshold of the candidate text. For specific comparison methods of threshold comparison conditions, please refer to the relevant descriptions in other embodiments.
[0088] In this embodiment, based on determining a first threshold corresponding to the candidate text according to the string length relationship between the query text and the candidate text, the target similarity between each candidate text and the query text can be calculated to obtain the target similarity corresponding to each candidate text. Target similarity measures the degree of similarity between the candidate text and the query text. By using the target similarity of the candidate text and the first threshold to dynamically determine whether the candidate text meets the threshold comparison condition, a more refined screening of the similarity between the candidate text and the query text is performed, resulting in a more accurate subset of target candidate texts. By performing a secondary screening of the target candidate text subset with the highest similarity, a double-refinement of the candidate texts is achieved, improving the accuracy of target text selection. Furthermore, using target candidate texts reduces the number of candidate texts participating in the highest similarity comparison, thus improving the efficiency of querying questions.
[0089] As an example, in step 401, calculating the target similarity between each candidate text and the query text includes:
[0090] Determine at least one candidate word segment corresponding to each candidate text;
[0091] For each candidate text, at least one candidate word is used to calculate the word similarity between the candidate word and the query text based on the number of times the candidate word appears in the query text, thereby obtaining the word similarity between at least one candidate word.
[0092] The target similarity of each candidate text is obtained by weighted summing of the word segmentation similarities corresponding to at least one candidate word for each candidate text.
[0093] Optionally, a word segmentation algorithm can be used to segment each candidate text to obtain at least one candidate word for each candidate text. Similarly, a word segmentation algorithm can be used to segment the query text to obtain at least one query word.
[0094] The number of times a candidate word appears in at least one query word can be the number of query words with the same characters as the candidate word identified from at least one query word. Each candidate word can correspond to a word similarity score.
[0095] In this embodiment, the target similarity can be calculated using at least one candidate word corresponding to the candidate text and at least one query word corresponding to the query text. The word segmentation similarity of the candidate word can be determined by the number of times each candidate word appears in at least one query word. Since the candidate word is a part of the candidate text, the target similarity of the candidate text can be obtained by weighted summing of the word segmentation similarities corresponding to at least one candidate word. This achieves a similarity calculation strategy from local to comprehensive, resulting in an accurate target similarity for the candidate text.
[0096] To obtain accurate word segmentation similarity, the word segmentation similarity of the candidate word relative to the query text is calculated based on the frequency of occurrence of the candidate word in at least one query word, thus obtaining the word segmentation similarity corresponding to each of the at least one candidate word. This may include:
[0097] For at least one candidate word segment, determine the total number of occurrences of the at least one candidate word segment in at least one query word;
[0098] The word segmentation similarity between the candidate word and the query text is calculated based on the number of times and the total number of times the candidate word appears in at least one query word.
[0099] Optionally, the total number of occurrences of at least one candidate word in at least one query word can be obtained by summing the occurrences of at least one candidate word in at least one query word.
[0100] Optionally, word segmentation similarity can be the similarity between candidate word segments and query text, which can be calculated by the number of times the candidate word segment appears in the query text. Specifically, it can be calculated by combining the number of times the candidate word segment appears and the total number of times at least one candidate word segment appears, along with a constant decimal. The sum of the total number of occurrences and the constant decimal can be used to obtain the target total number; the quotient of the number of times the candidate word segment appears and the target total number can then be used to obtain the word segmentation similarity of the candidate word segment relative to the query text.
[0101] The constant decimal can be a very small constant relative to the total number of occurrences; for example, it can be set to 0.01. Summing the total number of occurrences with the constant decimal can prevent the denominator from being zero when the total number is 0.
[0102] Let N represent the total number of occurrences, and fi represent the number of occurrences of any candidate word i. Then, the word segmentation similarity of the candidate words can be expressed as:
[0103] Where γ is a constant decimal.
[0104] In this embodiment, the segmentation similarity of a single candidate word can be determined based on the number of times the candidate word appears in at least one query word and the total number of occurrences of all candidate words. The number of occurrences of a word can represent the relevance of the word to the query text. Determining the segmentation similarity of candidate words using the number of occurrences allows for an accurate measurement of the similarity between candidate words and the query text, improving the computational efficiency and accuracy of candidate word segmentation.
[0105] To accurately calculate the weighted similarity of at least one candidate word, the word similarity scores corresponding to at least one candidate word in each candidate text are weighted and summed to obtain the target similarity score for each candidate text, including:
[0106] For each candidate text, calculate the segmentation weight of the candidate segment relative to the query text for at least one candidate segment, and obtain the segmentation weight corresponding to each of the at least one candidate segment.
[0107] Determine the indicator value corresponding to at least one candidate word; the indicator value is used to indicate whether the candidate word appears in the query text; multiply the indicator value of the candidate word by the corresponding word weight to obtain the target word weight corresponding to at least one candidate word; perform a weighted sum of the target word weights corresponding to at least one candidate word in each candidate text to obtain the target similarity of each candidate text.
[0108] The indicator value of the candidate word segmentation can indicate whether the candidate word segmentation appears in the query text. The indicator value can be used to confirm whether the candidate text participates in the weight calculation. The target word weight can be calculated quickly directly for the candidate word segmentation, thereby improving the calculation efficiency of the target word weight.
[0109] Each candidate word segment can correspond to an indicator value. For candidate word segment i, the indicator value can be expressed as I. i The target word weight of candidate word i can be represented using W. i *I i express.
[0110] The target similarity of candidate texts can be obtained by weighted summing of the target word weights and word similarity corresponding to at least one candidate word in the candidate text. Specifically, it can be calculated using the following formula:
[0111]
[0112] Where Sim(Q, A) represents the target similarity between the candidate text and the query text. Q is a set consisting of at least one query segmentation. A is a set consisting of at least one candidate segmentation, n is the number of segments in the at least one candidate segmentation, and i is the label of the candidate text.
[0113] In this embodiment, by multiplying the segmentation weight and indicator value of each candidate word, the occurrence association between the candidate text and the query text can be calculated to obtain the target word weight for each candidate word. The indicator value can be used to determine whether the candidate text participates in the weight calculation, enabling rapid calculation of the target word weight. When the segmentation similarity corresponding to at least one candidate word is weighted and summed, the target similarity of each candidate text can be obtained by combining the target word weights corresponding to at least one candidate word, thus improving the efficiency and accuracy of the target similarity calculation for each candidate text.
[0114] Furthermore, the segmentation weights of candidate words can be calculated based on a combination of part-of-speech weights and positional weights. Therefore, in one possible design, calculating the segmentation weights of candidate words relative to the query text includes:
[0115] The word segmentation weight of the candidate word relative to the query text is determined based on the positional weight of the candidate word in the query text and the part-of-speech weight of the candidate word.
[0116] In this embodiment, the positional weight and part-of-speech weight of candidate words in the query text are utilized to accurately calculate the word segmentation weight of candidate words relative to the query text. The word segmentation weight of candidate words relative to the query text comprehensively considers the positional weight and part-of-speech weight of candidate words, realizing the weight calculation of candidate words from multiple perspectives and improving the accuracy of the word segmentation weight.
[0117] To calculate word segmentation weights, the relative weights of candidate word segments to the query text are determined based on their positional weights and part-of-speech weights within the query text. This includes:
[0118] Determine the position weight and part-of-speech weight of the candidate word in the query text; calculate the product of the position weight and the word weight to obtain a first value; calculate the sum of the position weight and the word weight to obtain a second value; multiply the ratio of the first value and the second value by a preset first constant to obtain the word weight of the candidate word relative to the query text.
[0119] The word weight of candidate word i can be calculated using the following formula:
[0120]
[0121] In this formula, the first constant is 2. i t represents the positional weight of candidate word i. i Let be the part-of-speech weight of word i.
[0122] The first constant can be set according to usage requirements; in this embodiment, it is preferably set to 2.
[0123] p i *t i p is the first value. i +t i This is the second value.
[0124] In this embodiment of the application, when using the position weight and part-of-speech weight of the candidate word segmentation, the product of the position weight and the word segmentation weight and the ratio of the position weight and the word segmentation weight can be multiplied by a first constant to obtain the word segmentation weight of the candidate word segmentation relative to the query text, thereby obtaining accurate word segmentation weight.
[0125] The positional weight and word segmentation weight of candidate word segments can be determined by the position of the candidate word segment in the query text and the word type itself. Therefore, in some embodiments, before determining the word segmentation weight of the candidate word segment relative to the query text based on its positional weight and part-of-speech weight in the query text, the text matching method disclosed herein may further include:
[0126] The position weight of the candidate word in the query text is determined based on the starting position of the candidate word in the query text and the length of the first string of the query text.
[0127] Retrieve the part-of-speech weights corresponding to the word types of the candidate word segments from the part-of-speech weight list; the part-of-speech weight list includes the part-of-speech weights corresponding to at least one word type.
[0128] Optionally, the word type can be the nature of the candidate word segment in the query text. Different word types can be assigned different part-of-speech weights. For ease of understanding, the part-of-speech weights for different word types can be determined using the following list of part-of-speech weights:
[0129]
[0130] α1, α2…α m The part-of-speech weight of the word type corresponding to candidate word i (the nature of the word in the query text of candidate word i).
[0131] In the part-of-speech tagging list, the word properties that best reflect the semantics of the query text are more effective than other word properties; that is, the part-of-speech weight of entity words is greater than that of non-entity words. For example, nouns and gerunds are more effective at reflecting the main semantics of the query text than adjectives. Therefore, α1 and α2 are generally relatively large, and can take values of 0.9 or 0.8, while… α m-1 α m It can take values of 0.1 or 0.
[0132] In this embodiment, the position weight of a candidate word in the query text can be determined based on its starting position and the length of the first string of the query text, thus achieving accurate extraction of the position weight of the candidate word. The part-of-speech weight of the candidate word can be accurately calculated by querying the part-of-speech weight list corresponding to the word type of the candidate word.
[0133] Furthermore, the positional weight can be determined based on the position of the candidate word in the query text. The positional weight of the candidate word in the query text is determined based on its starting position and the length of the first string of the query text, and can include:
[0134] Calculate the ratio of the starting position of the candidate word in the query text to the length of the first string of the query text to obtain the third value; multiply the third value by pi to obtain the fourth value; calculate the sine of the fourth value to obtain the position weight of the candidate word in the query text.
[0135] Suppose the starting position of the candidate word in the query text is x, and the length of the first string of the query text is l.
[0136] For example, the position weight p of candidate word i i The following formula can be used to calculate it:
[0137]
[0138] in, It is the third value. It is the fourth value.
[0139] Position weights can be determined by the starting position of candidate words in the query text.
[0140] In this embodiment, the position weight of the candidate word is calculated by combining the starting position of the candidate word, the length of the first string, and pi with sine calculation, thereby achieving accurate calculation of the candidate word.
[0141] To achieve accurate association between candidate word segments and query text, in one possible design, determining an indicator value corresponding to at least one candidate word segment can include:
[0142] Iterate through at least one candidate word segment. If the candidate word segment has the same segment as at least one query word segment, then determine the second constant as the indicator value of the candidate word segment; otherwise, determine the third constant as the indicator value of the candidate word segment.
[0143] Optionally, candidate word segmentation can be matched with at least one query word to determine whether the query word is the same as the candidate word. If one or more query words are the same as the candidate word, the indicator value of the candidate word is determined to be a second constant. If no query word is the same as the candidate word, the indicator value of the candidate word is determined to be a third constant.
[0144] Optionally, the second constant can be 1, and the third constant can be 0. By multiplying 1 or 0 by the weights of the candidate words, the weights of the word similarity of candidate words that do not exist in the query text can be adjusted to 0 during the weighted calculation of the word similarity of the candidate words, thus achieving accurate weighted calculation of the candidate words.
[0145] In this embodiment, the indicator value of the candidate word can be determined by whether the candidate word exists in at least one query word. The indicator value can quickly indicate whether the candidate word appears in the query text, which helps improve the efficiency of candidate word weight calculation.
[0146] The selection of target candidate texts can be specifically determined based on the target similarity of candidate word segments and a first threshold. In some embodiments, target candidate texts that meet the threshold comparison conditions are determined from the candidate text set based on the target similarity corresponding to each candidate text and the first threshold, including:
[0147] Traverse the candidate text set. If the target similarity of any candidate text is greater than the corresponding first threshold, then the candidate text is determined to be a target candidate text that meets the threshold comparison condition, until the candidate text set is traversed to the end.
[0148] At the end of the candidate text set traversal, the target candidate texts that meet the threshold comparison conditions are obtained from the candidate text set.
[0149] If the target similarity of a candidate text is greater than the first threshold, then the candidate text is determined to be a target candidate text that meets the threshold comparison condition. If the target similarity of a candidate text is less than or equal to the first threshold, then the candidate text is determined not to meet the threshold comparison condition.
[0150] In this embodiment, the target similarity of candidate texts is compared with a first threshold. When the target similarity is greater than the first threshold, the candidate text is determined to meet the threshold comparison condition. The first threshold can be used to eliminate candidate texts that do not meet the threshold comparison condition, ensuring that all target candidate texts participating in subsequent feature ranking meet the threshold comparison condition, thus achieving accurate calculation of candidate texts.
[0151] For a detailed description of the technical solution of this application, such as Figure 5 The diagram shown is a flowchart of another embodiment of a text matching method provided in this application. The method may include the following steps:
[0152] 501: In response to a query request for query text, retrieve the query text provided by the target user.
[0153] 502: Based on the ES engine, query the question-answering database for a set of candidate texts that meet similar conditions to the query text; the candidate text set includes at least one candidate text.
[0154] 503: Based on the string length of the query text and the string length of each candidate text in the candidate text set, determine the first threshold corresponding to each candidate text. The first threshold is used to measure the similarity between the corresponding candidate text and the query text.
[0155] 504: Calculate the target similarity between each candidate text and the query text to obtain the target similarity corresponding to each candidate text.
[0156] 505: Based on the target similarity and the first threshold corresponding to each candidate text, determine the target candidate text subset that meets the threshold comparison condition from the candidate text set.
[0157] 506: Identify the target text with the highest similarity to the query text from the subset of target candidate texts.
[0158] 507: Determine the target answer associated with the target text, which is used to provide feedback to the target user.
[0159] Optionally, if it is determined that the target candidate text subset is empty, that is, the number of target candidate texts in the target candidate text subset is zero, an input prompt message can be output to prompt the target user to re-enter the query text.
[0160] Optionally, determining the target text with the highest similarity to the query text from the target candidate text subset may include: if the number of target candidate texts in the determined target candidate text subset is greater than zero, then determining the target text with the highest similarity to the query text from the target candidate text subset.
[0161] In this embodiment, for a query request containing query text, the query text provided by the target user can be obtained. After querying a set of candidate texts that meet similarity criteria with the query text from the question-and-answer database using the Elasticsearch engine, a first threshold can be determined for each candidate text based on the string length relationship between the query text and each candidate text. This first threshold represents the similarity measurement used for the corresponding candidate text. After obtaining the target similarity for each candidate text, the first threshold for each candidate text can be compared with the target similarity to determine whether each candidate text meets the threshold comparison criteria, thus identifying a subset of target candidate texts that meet the threshold comparison criteria from the candidate text set. The first threshold is used to more precisely filter the candidate text set, obtaining the target candidate text subset. This obtained subset of target candidate texts can then be filtered again, achieving double ranking of candidate texts and improving the accuracy of target text selection. Furthermore, using target candidate texts reduces the number of candidate texts participating in the highest similarity comparison, thus improving query efficiency.
[0162] like Figure 6 The diagram shown is a structural schematic of one embodiment of a text matching device provided in this application. This text matching device can execute a text matching method and can be located in an electronic device. The text matching device 600 may include:
[0163] Candidate determination unit 601: In response to a query request for a query text, it queries a set of candidate texts that meet similar conditions from a question-and-answer database. The set of candidate texts includes at least one candidate text.
[0164] Threshold determination unit 602: is used to determine a first threshold corresponding to each candidate text based on the string length of the query text and the string length of each candidate text in the candidate text set. The first threshold is used to measure the similarity between the corresponding candidate text and the query text.
[0165] Threshold comparison unit 603: used to determine a subset of target candidate texts that meet the threshold comparison conditions from the candidate text set using the first threshold corresponding to each candidate text.
[0166] Target determination unit 604: used to determine the target text with the highest similarity to the query text from the subset of target candidate texts.
[0167] As one embodiment, the threshold determination unit includes:
[0168] The length determination module is used to obtain the string ratio of the first string length and the second string length based on the first string length of the query text and the second string length of the corresponding candidate text.
[0169] The threshold determination module is used to determine the first threshold of the candidate text based on the string ratio and the preset target value.
[0170] In some embodiments, the threshold determination module includes:
[0171] The first determination submodule is used to determine the first threshold of the candidate text by multiplying the preset initial threshold and the preset first threshold coefficient if the string ratio is greater than or equal to the target value.
[0172] The second determining submodule is used to determine the first threshold of the candidate text based on the initial threshold and the string ratio if the string ratio is less than the target value.
[0173] In some embodiments, the second determining submodule is specifically used for:
[0174] The second threshold coefficient is calculated based on the string ratio; the second threshold coefficient is used to indicate the similarity comparison threshold related to the string length of the candidate text.
[0175] Calculate the product of the second threshold coefficient and the initial threshold to obtain the first threshold of the candidate text.
[0176] In one possible design, the second determining submodule can be specifically used for:
[0177] Calculate the product of the string ratio and pi to obtain the first ratio data;
[0178] Calculate the quotient of the first ratio data and the preset first constant to obtain the second ratio data;
[0179] Calculate the cosine value of the second ratio data to obtain the second threshold coefficient.
[0180] As another embodiment, the candidate text includes at least one, each candidate text includes a target similarity and a first threshold, and the above-described apparatus further includes:
[0181] The similarity calculation unit is used to calculate the similarity between each candidate text and the query text, and obtain the target similarity corresponding to each candidate text;
[0182] The threshold comparison unit may include:
[0183] The threshold comparison module is used to determine the target candidate texts that meet the threshold comparison conditions from the candidate text set based on the target similarity corresponding to each candidate text and the first threshold.
[0184] The subset determination module is used to determine a subset of target candidate texts based on the target candidate texts.
[0185] In some embodiments, the similar computing unit may include:
[0186] The word segmentation module is used to determine at least one candidate word for each candidate text.
[0187] The similarity calculation module is used to calculate the word similarity between the candidate word and the query text based on the number of times the candidate word appears in the query text for each candidate text, and to obtain the word similarity corresponding to each candidate word.
[0188] The weighted summation module is used to sum the word similarity scores corresponding to at least one candidate word for each candidate text in a weighted manner to obtain the target similarity score for each candidate text.
[0189] In some embodiments, the similar computing module may include:
[0190] The total count determination submodule is used to determine the total number of occurrences of at least one candidate word in at least one query word for at least one candidate word.
[0191] The similarity calculation submodule is used to calculate the word similarity between the candidate word and the query text based on the number of times and the total number of times the candidate word appears in at least one query word.
[0192] In some embodiments, the weighted summation module includes:
[0193] The weight determination submodule is used to calculate the word segmentation weight of each candidate word relative to the query text for each candidate text and to obtain the word segmentation weight corresponding to each candidate word.
[0194] The indicator determination submodule is used to determine the indicator value corresponding to at least one candidate word; the indicator value is used to indicate whether the candidate word appears in the query text.
[0195] The first determining submodule is used to multiply the indicator value of the candidate word segment by the corresponding word segmentation weight to obtain the target word weight corresponding to at least one candidate word segmentation.
[0196] The similarity calculation submodule is used to perform a weighted summation of the target word weights corresponding to at least one candidate word segment of each candidate text to obtain the target similarity of each candidate text.
[0197] In some embodiments, the weight determination submodule can specifically be used for:
[0198] The word segmentation weight of the candidate word relative to the query text is determined based on the positional weight of the candidate word in the query text and the part-of-speech weight of the candidate word.
[0199] In some embodiments, the weight determination submodule can specifically be used for:
[0200] Determine the positional weight of candidate words in the query text and the part-of-speech weight of candidate words;
[0201] Calculate the product of the position weight and the word segmentation weight to obtain the first value;
[0202] Calculate the sum of the position weight and the word segmentation weight to obtain the second value;
[0203] Multiply the ratio of the first and second values by a preset first constant to obtain the word segmentation weight of the candidate word relative to the query text.
[0204] One possible design also includes:
[0205] The position weight unit is used to determine the position weight of the candidate word in the query text based on the starting position of the candidate word in the query text and the length of the first string of the query text.
[0206] The part-of-speech weight unit is used to query the part-of-speech weights corresponding to the word types of candidate words from the part-of-speech weight list; the part-of-speech weight list includes the part-of-speech weights corresponding to at least one word type.
[0207] In one possible design, the location weighting unit includes:
[0208] The second calculation module is used to calculate the ratio of the starting position of the candidate word in the query text to the length of the first string of the query text, and obtain the third value;
[0209] The third calculation module is used to multiply the third value by pi to obtain the fourth value;
[0210] The fourth calculation module is used to calculate the sine value of the fourth value to obtain the position weight of the candidate word in the query text.
[0211] At least one embodiment indicates that the determining submodule can specifically be used for:
[0212] Iterate through at least one candidate word segment. If the candidate word segment has the same segment as at least one query word segment, then determine the second constant as the indicator value of the candidate word segment; otherwise, determine the third constant as the indicator value of the candidate word segment.
[0213] In some embodiments, the threshold comparison module includes:
[0214] The problem comparison submodule is used to traverse the candidate text set. If the target similarity of any candidate text is greater than the corresponding first threshold, then the candidate text is determined to be a target candidate text that meets the threshold comparison condition, until the candidate text set is traversed to the end.
[0215] The second determination submodule is used to obtain the target candidate texts that meet the threshold comparison conditions in the candidate text set when the candidate text set traversal is completed.
[0216] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. The device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.
[0217] Electronic device 700 may include one or more of the following components: processing component 702, memory 704, power supply component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 714, and communication component 716.
[0218] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.
[0219] Memory 704 is configured to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0220] Power supply component 706 provides power to various components of electronic device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.
[0221] Multimedia component 708 includes a screen that provides an output interface between electronic device 700 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When electronic device 700 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0222] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.
[0223] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0224] Sensor assembly 714 includes one or more sensors for providing state detection of various aspects of electronic device 700. For example, sensor assembly 714 can detect the on / off state of electronic device 700, the relative positioning of components, such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.
[0225] Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0226] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0227] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of an electronic device 700 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0228] This application provides a non-transitory computer-readable storage medium, wherein when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the above-described text matching method.
[0229] In addition, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the text matching method in the above embodiments.
[0230] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0231] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A text matching method, characterized in that, include: In response to a query request for a query text, a set of candidate texts that meet similar conditions is retrieved from a question-and-answer database, wherein the set of candidate texts includes at least one candidate text. Based on the string ratio of the query text's string length to the string length of each candidate text in the candidate text set, a first threshold is determined for each candidate text; wherein, the first threshold is used to measure the similarity between the corresponding candidate text and the query text; when the string ratio is greater than or equal to a preset target value, the first threshold is determined based on a preset initial threshold and a preset first threshold coefficient; when the string ratio is less than the preset target value, the first threshold is determined based on the initial threshold and the string ratio. Using the first threshold corresponding to each candidate text, a subset of target candidate texts that meet the threshold comparison conditions is determined from the candidate text set; The target text with the highest similarity to the query text is determined from the subset of target candidate texts.
2. The method according to claim 1, characterized in that, The step of determining the first threshold based on the initial threshold and the string ratio includes: A second threshold coefficient is calculated based on the string ratio; the second threshold coefficient is used to indicate the similarity comparison threshold related to the string length of the candidate text. The first threshold of the candidate text is obtained by multiplying the second threshold coefficient by the initial threshold.
3. The method according to claim 1 or 2, characterized in that, The method further includes: Calculate the similarity between each candidate text and the query text to obtain the target similarity corresponding to each candidate text; The step of determining a subset of target candidate texts that meet the threshold comparison conditions from the candidate text set using a first threshold corresponding to each candidate text includes: Based on the target similarity and first threshold corresponding to each candidate text, target candidate texts that meet the threshold comparison conditions are determined from the candidate text set. Based on the target candidate text, a subset of the target candidate text is determined.
4. The method according to claim 3, characterized in that, The calculation of the target similarity between each candidate text and the query text includes: Determine at least one candidate word segment corresponding to each candidate text; For each candidate text, at least one candidate word is used to calculate the word segmentation similarity between the candidate word and the query text based on the number of times the candidate word appears in the query text, thereby obtaining the word segmentation similarity corresponding to at least one candidate word. The target similarity of each candidate text is obtained by weighted summing of the word segmentation similarities corresponding to at least one candidate word of each candidate text.
5. The method according to claim 4, characterized in that, The step of weighted summing of the word segmentation similarities corresponding to at least one candidate word for each candidate text to obtain the target similarity of each candidate text includes: For each candidate word in the candidate text, calculate the word segmentation weight of the candidate word relative to the query text to obtain the word segmentation weight corresponding to each of the at least one candidate word; Determine an indicator value corresponding to at least one of the candidate words; the indicator value is used to indicate whether the candidate word appears in the query text. Multiply the indicator value of the candidate word by the corresponding word weight to obtain the target word weight corresponding to at least one of the candidate words; The target similarity of each candidate text is obtained by weighted summing of the target word weights corresponding to at least one candidate word segment of each candidate text.
6. The method according to claim 5, characterized in that, The calculation of the segmentation weight of the candidate word relative to the query text includes: The word segmentation weight of the candidate word relative to the query text is determined based on the position weight of the candidate word in the query text and the part-of-speech weight of the candidate word.
7. A text matching device, characterized in that, include: A candidate determination unit is configured to, in response to a query request for a query text, query a set of candidate texts that meet similar conditions from a question-and-answer database, wherein the set of candidate texts includes at least one candidate text. A threshold determination unit is configured to determine a first threshold for each candidate text based on the string ratio of the string length of the query text to the string length of each candidate text in the candidate text set; wherein, the first threshold is used to measure the similarity between the corresponding candidate text and the query text; when the string ratio is greater than or equal to a preset target value, the first threshold is determined based on a preset initial threshold and a preset first threshold coefficient; when the string ratio is less than the preset target value, the first threshold is determined based on the initial threshold and the string ratio. A threshold comparison unit is used to determine a subset of target candidate texts that meet the threshold comparison conditions from the candidate text set using a first threshold corresponding to each candidate text. The target determination unit is used to determine the target text with the highest similarity to the query text from the subset of target candidate texts.
8. An electronic device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the text matching method as described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the text matching method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Method and device for determining core statements of texts
CN107797990A
Text similarity recognition method and device, electronic equipment and readable storage medium
CN114692594A