Method for determining ambiguous words, search method, and electronic device

Abstract

The application relates to the technical field of data retrieval, in particular to a method for determining an ambiguous word, a retrieval method and an electronic device. The method for determining the ambiguous word comprises the following steps: obtaining an initial subject word; based on the initial subject word, performing segmentation and splicing on a preset text to obtain a target subject word containing the initial subject word; wherein the target subject word is segmented in a first segmentation mode to obtain a plurality of words containing the initial subject word, and is segmented in a second segmentation mode to obtain a plurality of continuous non-initial subject words; and if the number of times that the target subject word appears in the preset text is greater than a preset value, the target subject word is determined as an ambiguous word. In this way, based on the scheme provided in the application, the ambiguous word of the initial subject word can be automatically identified, and compared with the manual determination scheme in the background art, the scheme provided in the application is more time-saving and labor-saving.

Description

Technical Field

[0001] This application relates to the field of data retrieval technology, and in particular to a method for determining ambiguous words, a retrieval method, and an electronic device. Background Technology

[0002] In certain specific business scenarios, customers use keywords related to their own company, products, and personnel as search terms to index relevant data in the database, so as to accurately recall text data closely related to the customer's ecosystem and extract valuable information from the data.

[0003] However, during the retrieval process based on subject-term matching data, noise interference exists, affecting the accuracy of the retrieved data. This noise interference refers to the interference caused by ambiguous subject-term terms on the retrieval results. Ambiguous terms are those unrelated to the subject-term term but mistakenly considered closely related during retrieval. For example, when searching for "CCB" as the subject-term term, although "infrastructure industry" consists of two consecutive words "infrastructure" and "industry," because "infrastructure industry" includes "CCB," "infrastructure industry" will be mistakenly considered closely related to the subject-term term, affecting the search results. In this case, "infrastructure industry" is an ambiguous term for "CCB."

[0004] During retrieval, in order to eliminate the interference of these noisy data, it is necessary to manually determine the ambiguous words of the main words, but this method of manually determining the ambiguous words of the main words is time-consuming and laborious. Summary of the Invention

[0005] To at least partially overcome the time-consuming and laborious problem of manually determining ambiguous words in related technologies, this application provides a method for determining ambiguous words, a retrieval method, and an electronic device.

[0006] The proposed solution is as follows:

[0007] According to a first aspect of the embodiments of this application, a method for determining ambiguous words is provided, comprising:

[0008] Obtain the initial subject words;

[0009] Based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words containing the initial subject words; wherein, when the target subject words are segmented according to the first segmentation method, multiple words including the initial subject words are obtained, and when segmented according to the second segmentation method, multiple consecutive non-initial subject words are obtained;

[0010] If the target word appears more times in the preset text than a preset value, then the target word is determined to be an ambiguous word.

[0011] In some embodiments, based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words containing the initial subject words, including:

[0012] Locate the initial main words in the preset text;

[0013] For each initial subject word in the preset text, extract a first preset number of words before the initial subject word to a second preset number of words after the initial subject word to obtain the suspected subject word;

[0014] If the suspected subject word consists of multiple consecutive non-initial subject words, then the suspected subject word is the target subject word.

[0015] In some embodiments, including:

[0016] The first preset quantity is a non-negative integer, the second preset quantity is a non-negative integer, and the first preset quantity and the second preset quantity are not both zero at the same time.

[0017] In some embodiments, the initial subject word comprises a first part and a second part arranged in sequence;

[0018] Based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words that contain the initial subject words, including:

[0019] Identify the first type of words in the preset text that end with the first part and the second type of words that begin with the second part;

[0020] By combining a first-category word and a second-category word, the target subject word can be obtained.

[0021] According to a second aspect of the embodiments of this application, a retrieval method is provided, comprising:

[0022] Retrieve search terms;

[0023] Using the search terms as initial subject terms, ambiguous terms are determined based on the method provided in any embodiment of this application;

[0024] The ambiguous words are treated as search noise, and the search terms are searched to obtain search results.

[0025] According to a third aspect of the embodiments of this application, a retrieval method is provided, comprising:

[0026] Retrieve search terms;

[0027] Based on a preset database, obtain the ambiguous words corresponding to the search terms;

[0028] The preset database stores search terms and ambiguous words corresponding to the search terms determined by the method described above.

[0029] The ambiguous words are treated as search noise, and the search terms are searched to obtain search results.

[0030] In some embodiments, it also includes:

[0031] At preset time intervals, the search terms and their corresponding ambiguous terms stored in the preset database are updated.

[0032] In some embodiments, it also includes:

[0033] The search terms and their corresponding ambiguous terms are displayed for relevant personnel to view;

[0034] Obtain deletion instructions input by relevant personnel;

[0035] Based on the deletion instruction, ambiguous words corresponding to the search terms are deleted.

[0036] In some embodiments, displaying the search terms and their corresponding ambiguous terms includes:

[0037] If the same search term corresponds to multiple ambiguous words, the ambiguous words are sorted from most frequent to least frequent in the preset text.

[0038] The ambiguous words are displayed according to the principle of prioritizing the ambiguous words that appear most frequently in the preset text.

[0039] According to a fourth aspect of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method for determining ambiguous words provided in any of the above embodiments, or to implement the steps of the retrieval method provided in any of the above embodiments.

[0040] The technical solution provided in this application involves: firstly, obtaining initial subject words; then, based on the characteristic that ambiguous words often contain the initial subject words and that ambiguous words are composed of multiple non-initial subject words, segmenting and concatenating the preset text to obtain target subject words containing the initial subject words; wherein, when the target subject words are segmented according to the first segmentation method, multiple words including the initial subject words are obtained; when segmented according to the second segmentation method, multiple consecutive non-initial subject words are obtained; at this point, the target subject words are likely ambiguous, but if the frequency of the target subject words is low, it will not significantly interfere with data retrieval. Therefore, the target subject words with low frequency of occurrence are not ambiguous. Based on this, if the frequency of the target subject words in the preset text is greater than a preset value, then the target subject words are determined to be ambiguous. With this setup, based on the solution provided in this application, ambiguous words of the initial subject words can be automatically identified. Compared to the manual determination method in the background technology, the solution provided in this application is more time-saving and labor-saving.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] Figure 1 This is a flowchart illustrating a method for determining ambiguous words according to an embodiment of this application;

[0044] Figure 2 This is a partial flowchart illustrating a method for determining ambiguous words according to an embodiment of this application;

[0045] Figure 3 This is a partial flowchart illustrating a method for determining ambiguous words according to an embodiment of this application;

[0046] Figure 4 This is a flowchart illustrating a retrieval method provided in one embodiment of this application;

[0047] Figure 5 This is a flowchart illustrating a retrieval method provided in one embodiment of this application;

[0048] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0050] Example 1

[0051] Figure 1 This is a flowchart illustrating a method for determining ambiguous words provided in one embodiment of this invention. (Refer to...) Figure 1 Methods for identifying ambiguous words include:

[0052] S101: Obtain the initial main words;

[0053] It should be noted that the initial subject term here is the word entered by the user to be searched. For example, in some cases, if it is necessary to search for "CCB", then "CCB" can be used as the initial subject term.

[0054] It should be noted that noise interference is more likely to occur when the initial subject word is short. In the scheme provided in this application, the initial subject word mainly refers to a word consisting of two or three characters.

[0055] S102: Based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words containing the initial subject words; wherein, when the target subject words are segmented according to the first segmentation method, multiple words including the initial subject words are obtained, and when segmented according to the second segmentation method, multiple consecutive non-initial subject words are obtained;

[0056] Specifically, when searching using "CCB" as the subject term, the target subject term "infrastructure industry" containing the initial subject term "CCB" can be found. Furthermore, when "infrastructure industry" is segmented according to the first segmentation method, multiple words including the initial subject term "infrastructure," "CCB," and "industry" are obtained; when "infrastructure industry" is segmented according to the second segmentation method, multiple consecutive non-initial subject terms "infrastructure" and "industry" are obtained.

[0057] S103: If the number of times the target word appears in the preset text is greater than a preset value, then the target word is determined to be an ambiguous word.

[0058] Furthermore, in actual searches, target words that appear too infrequently will not affect the search results. Based on this, in the solution provided in this application, the target word is determined to be an ambiguous word only when the number of times the target word appears in the preset text is greater than a preset value.

[0059] With this setup, based on the solution provided in this application, ambiguous words in the initial subject can be automatically identified. Compared with the manual determination method in the background technology, the solution provided in this application is more time-saving and labor-saving.

[0060] The solution provided in this application will be illustrated below with a specific application scenario:

[0061] When helping clients retrieve relevant text, operations personnel typically use main keywords and related ecosystem keywords, such as company name, product name, and executive names, as matching words to retrieve large amounts of data from streaming data. After further processing, this data is pushed to clients. However, in practice, each client's main keywords often match text data that does not belong to the client's industry or ecosystem in the retrieved data results. This invention directly uses the input fine-grained main keywords to perform co-occurrence statistics, calculating from multiple dimensions to retrieve potentially large-grained ambiguous word groups. This information is then provided to operations personnel for screening. This essentially delegates the process of manually browsing data to find ambiguous word groups to machine computation. Subsequently, operations personnel only need to perform simple filtering on the calculation results. At this point, the data retrieved through main keyword matching will no longer contain irrelevant text noise, thus improving the efficiency of data push.

[0062] Specifically, based on the initial subject words, the preset text is segmented and concatenated to obtain the target subject words containing the initial subject words, as shown in the following example. Figure 2 It may include the following steps:

[0063] S201, Locate the initial subject word in the preset text;

[0064] Specifically, in practical applications, the way to locate the initial subject words in a preset text is to search the preset text to find all the initial subject words in the preset text.

[0065] S202, for each initial subject word in the preset text, extract a first preset number of words before the initial subject word to a second preset number of words after the initial subject word to obtain a suspected subject word;

[0066] It should be noted that the first and second preset quantities are not fixed. They can be set based on the actual situation. For example, when the initial subject word is a two-character subject word, the first prediction quantity can be set to 1, and the second prediction quantity can also be set to 1. In this case, the extracted suspected subject word will be a four-character suspected subject word, i.e., the suspected subject word will be in the form of 'ABBC' (A (previous character) + BB (subject word) + C (last character)).

[0067] S203, if the suspected subject word consists of multiple consecutive non-initial subject words, then the suspected subject word is the target subject word.

[0068] Specifically, a professional word segmentation tool can be used to segment the suspected subject word in the form 'ABBC'. If two independent words, 'AB' and 'BC', are obtained, then the suspected subject word is considered the target subject word. If 'AB' and 'BC' are not separate words after segmentation, then the suspected subject word is not considered the target subject word.

[0069] It should be noted that the first preset quantity is a non-negative integer, the second preset quantity is a non-negative integer, and the first preset quantity and the second preset quantity are not both zero.

[0070] That is, the values ​​of the first preset quantity and the second preset quantity can be as mentioned in the above embodiments. The value of the first preset quantity is 1, and the value of the second preset quantity is also 1.

[0071] Furthermore, when the initial subject word has 3 characters, the first preset number can be 1 and the second preset number can be 0; alternatively, the first preset number can be 0 and the second preset number can be 1. In this case, the resulting suspected subject word will also be a 4-character suspected subject word. Specifically, when using a professional word segmentation tool to segment this type of suspected subject word, if it is cut in the middle and the first two characters can form a word, and the last two characters can also form a word, then the suspected subject word is considered the target subject word. If, after segmentation, the first two characters can form a word and the last two characters can also form a word, but they are not separate words, then the suspected subject word is considered not the target subject word.

[0072] Furthermore, in practical applications, the initial subject words include a first part and a second part arranged sequentially; based on this, refer to Figure 3 Based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words that contain the initial subject words, including:

[0073] S301, determine the first type of words in the preset text that end with the first part and the second type of words that begin with the second part;

[0074] S302, combine a first-class word and a second-class word to obtain the target subject word.

[0075] For example, "CCB" can be broken down into "CCB" and "CAB". Then, in the pre-defined text, first-category words ending with "CCB", such as "infrastructure", "team building", and "housing construction", are identified; and second-category words beginning with the second part, such as "behavior", "action", and "administration". Concatenating these words yields target subject terms such as "infrastructure behavior", "infrastructure action", "infrastructure administration", "team building behavior", "team building action", "team building administration", "housing construction behavior", "housing construction action", and "housing construction administration".

[0076] This setup allows for the determination of target keywords from multiple dimensions, resulting in a more comprehensive and accurate selection of target keywords.

[0077] Furthermore, after obtaining the target keywords, a screening process is performed. Words that appear too infrequently among the target keywords are identified as ambiguous words.

[0078] Specifically, the above screening process can be based on the number of times the target subject word appears, or it can be based on the frequency of the target subject word's appearance.

[0079] Example 2

[0080] This application also provides a retrieval method, referring to... Figure 4 ,include:

[0081] S401, retrieve search terms;

[0082] Specifically, the search terms are entered by the user.

[0083] S402, using the search terms as initial subject terms, determine the ambiguous terms based on the method for determining ambiguous terms provided in this application;

[0084] S403, the ambiguous words are treated as search noise, and the search terms are searched to obtain search results.

[0085] With this setup, no manual intervention is needed to determine ambiguous terms during retrieval, reducing the time required for manual identification and making the retrieval process more convenient and faster.

[0086] Furthermore, embodiments of this application also provide a retrieval method, referring to... Figure 5 include:

[0087] S601, retrieve search terms;

[0088] Specifically, the search terms are entered by the user;

[0089] S602, Based on a preset database, obtain the ambiguous words corresponding to the search terms;

[0090] The preset database stores search terms and ambiguous terms corresponding to the search terms, determined by the above-described method for determining ambiguous terms.

[0091] It should be noted that the solution provided in this application preprocesses a large number of words that may be used as search terms based on the aforementioned method for determining ambiguous terms, resulting in a large number of words and their ambiguous terms, which are then stored in a pre-defined database. During actual retrieval, the ambiguous terms corresponding to the search terms can be obtained directly by accessing the information stored in the database.

[0092] For example, the database may pre-store terms such as "infrastructure" and their corresponding ambiguous terms. If a term like "infrastructure" is used as a search term during a retrieval process, then the ambiguous terms for "infrastructure" can be directly retrieved from the database.

[0093] S603, the ambiguous words are treated as search noise, and the search terms are searched to obtain search results.

[0094] With this setup, no manual intervention is needed to determine ambiguous terms during retrieval, reducing the time required for manual identification and making the retrieval process more convenient and faster.

[0095] In practical applications, as time changes, the ambiguous words stored in the database may no longer match the ambiguous words that currently have a significant impact on retrieval. Based on this, the solution provided in this application also includes:

[0096] At preset time intervals, the search terms and their corresponding ambiguous terms stored in the preset database are updated.

[0097] The vocabulary in the database can be updated so that the ambiguous words retrieved from the database are more consistent with the current situation.

[0098] Furthermore, the ambiguous words identified in the provided solution are highly dependent on computers and pre-defined rules. However, in practical applications, relying solely on the above methods may result in some errors in the obtained ambiguous words. Therefore, the solution provided in this application also includes:

[0099] The search terms and their corresponding ambiguous terms are displayed for relevant personnel to view;

[0100] Obtain deletion instructions input by relevant personnel;

[0101] Based on the deletion instruction, ambiguous words corresponding to the search terms are deleted.

[0102] This setup allows for further deletion of ambiguous words based on instructions sent by relevant personnel, resulting in the most suitable ambiguous words.

[0103] Specifically, the process of displaying the search term and its corresponding ambiguous words includes: if the same search term corresponds to multiple ambiguous words, sorting each ambiguous word according to the number of times it appears in the preset text from most to least; and displaying each ambiguous word according to the principle that the ambiguous word that appears more often in the preset text appears first.

[0104] It should be noted that in practical applications, the ambiguous words can be sorted in advance based on their weights, and when displaying ambiguous words, the ones that appear more frequently will be displayed first.

[0105] Example 3

[0106] Figure 6 This is a schematic block diagram of an electronic device provided in one embodiment of this invention, with reference to... Figure 6 An electronic device includes a memory 72 and a processor 71, wherein the memory 72 stores a computer program, characterized in that the processor 71 executes the computer program as a step of the method for determining ambiguous words provided in any of the above embodiments, or as a step of implementing the retrieval method provided in any of the above embodiments.

[0107] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0108] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0109] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0110] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0114] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0115] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for determining ambiguous words, characterized in that, include: Obtain initial subject words, which are subject words input by the user for text retrieval or text recall; Based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words containing the initial subject words; wherein, when the target subject words are segmented according to the first segmentation method, multiple words including the initial subject words are obtained, and when segmented according to the second segmentation method, multiple consecutive non-initial subject words are obtained; If the target word appears more times in the preset text than a preset value, the target word is determined to be an ambiguous word. Based on the ambiguous word, a preset database is constructed to assist in retrieval. Based on the initial subject words, the preset text is segmented and concatenated to obtain target subject words that contain the initial subject words, including: Locating the initial subject word in the preset text includes searching the preset text to determine all occurrence positions of the initial subject word in the preset text; for each initial subject word in the preset text, extracting a first preset number of characters before the initial subject word to a second preset number of characters after the initial subject word to obtain a suspected subject word; wherein, the first preset number is a non-negative integer, the second preset number is a non-negative integer, and the first preset number and the second preset number are not both zero; The suspected subject words are segmented. If each non-initial subject word obtained after segmenting the suspected subject words according to the second segmentation method is an independent word, then the suspected subject words consist of multiple consecutive non-initial subject words, and the suspected subject words are determined as the target subject words; or, The initial subject words include a first part and a second part arranged in sequence; determine the first type of words in the preset text that end with the first part and the second type of words that begin with the second part; concatenate a first type of word and a second type of word to obtain the target subject words.

2. A retrieval method, characterized in that, include: Retrieve search terms; Using the search terms as initial subject terms, ambiguous words are determined based on the method described in claim 1; The ambiguous words are treated as search noise, and the search terms are searched to obtain search results.

3. A retrieval method, characterized in that, include: Retrieve search terms; Based on a preset database, obtain the ambiguous words corresponding to the search terms; The preset database stores search terms and ambiguous words corresponding to the search terms determined based on the method described in claim 1. The ambiguous words are treated as search noise, and the search terms are searched to obtain search results.

4. The retrieval method according to claim 3, characterized in that, Also includes: At preset time intervals, the search terms and their corresponding ambiguous terms stored in the preset database are updated.

5. The retrieval method according to claim 3, characterized in that, Also includes: The search terms and their corresponding ambiguous terms are displayed for relevant personnel to view; Obtain deletion instructions input by relevant personnel; Based on the deletion instruction, ambiguous words corresponding to the search terms are deleted.

6. The retrieval method according to claim 5, characterized in that, Display the search terms and their corresponding ambiguous terms, including: If the same search term corresponds to multiple ambiguous words, the ambiguous words are sorted from most frequent to least frequent in the preset text. The ambiguous words are displayed according to the principle of prioritizing the ambiguous words that appear most frequently in the preset text.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for determining ambiguous words as described in claim 1, or the steps of the retrieval method as described in any one of claims 2 to 6.

Images