Data processing method and device, and computer storable medium
By calculating the search degree of electronic documents and selecting search keywords with high missed frequency, the problem of low search engine sorting efficiency in the existing technology is solved, the document matching degree and search efficiency are improved, and the search habit needs of different users are met. .
Patent Information
- Application Number
- CN202010669313.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-07-13
AI Technical Summary
When existing search engines sort electronic documents, according to the search popularity strategy, the search results of different users are inefficient and cannot effectively match the search user's habits.
By obtaining the historical search data of electronic documents, calculating their search degree, selecting search keywords with high missed frequency as reference keywords, and recommending them as part of the document title or text to improve the matching degree and search efficiency of the document.
It improves the quality and accuracy of electronic documents, enhances the matching degree of search results, improves search efficiency, and meets the search habit needs of different users.
Smart Images

Figure CN113935316B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method and device, and a computer storable medium. Background Art
[0002] In the related art, editing users usually edit electronic documents according to their own understanding, and obtain a document database including multiple electronic documents. When searching the document library including multiple electronic documents according to the search terms input by the search user, the search engine sorts the multiple electronic documents in the document database according to the search popularity, and adjusts the electronic documents frequently searched by the search user to the beginning of the search results, so as to search for electronic documents matching the search terms as soon as possible. Summary of the invention
[0003] The inventor believes that: in the related art, different search users have different search habits such as the search terms they input, and thus different search users frequently search for different electronic documents, which results in the search engine adopting a strategy of sorting electronic documents according to search popularity to improve search efficiency. The effect is not obvious, and the efficiency of searching for electronic documents is still low.
[0004] In view of the above technical problems, the present disclosure proposes a solution to improve the quality and accuracy of electronic documents, thereby improving the efficiency of searching for electronic documents.
[0005] According to a first aspect of the present disclosure, there is provided a data processing method, comprising: acquiring historical search data corresponding to an electronic document, the historical search data comprising a plurality of search keywords; determining the searchability of the electronic document based on the historical search data, the searchability representing whether the electronic document is easy to be searched; selecting at least one reference keyword from the plurality of search keywords based on the searchability; and recommending the at least one reference keyword as a part of at least one of a title and a body of the electronic document.
[0006] In some embodiments, determining the searchability of the electronic document includes: determining, from the multiple search keywords, a plurality of first keywords that do not hit the electronic document and a plurality of second keywords that hit the electronic document, respectively, the first keywords being search keywords among the multiple search keywords that are not located in the title and text of the electronic document, and the second keywords being search keywords among the multiple search keywords that are located in at least one of the title and text of the electronic document; calculating, based on the historical search data, the miss frequency of each first keyword and the hit frequency of each second keyword; and determining the searchability based on the miss frequencies of the multiple first keywords and the hit frequencies of the multiple second keywords.
[0007] In some embodiments, selecting at least one reference keyword from the multiple search keywords includes: obtaining multiple first keywords from the multiple search keywords that do not hit the electronic document when the search ease is less than or equal to a threshold; obtaining the miss frequency of the multiple first keywords; and selecting at least one reference keyword from the multiple first keywords based on the ranking results of the miss frequencies of the multiple first keywords.
[0008] In some embodiments, selecting at least one reference keyword from the multiple first keywords includes: selecting the at least one reference keyword from N first keywords with the highest unhit frequency, where N is a positive integer.
[0009] In some embodiments, the searchability includes title searchability, the title searchability represents whether the title of the electronic document is easy to search, the threshold includes a title threshold, the multiple first keywords include multiple first title keywords that miss the title of the electronic document, and the miss frequency includes a title miss frequency.
[0010] In some embodiments, selecting at least one reference keyword from the multiple first keywords includes: selecting the at least one reference keyword from M first keywords with the highest title miss frequency and hitting the body of the electronic document, where M is a positive integer.
[0011] In some embodiments, the searchability includes the text searchability, the text searchability represents whether the text of the electronic document is easy to search, the threshold includes the text threshold, the multiple first keywords include multiple first text keywords that do not hit the text of the electronic document, and the miss frequency includes the text miss frequency.
[0012] In some embodiments, selecting at least one reference keyword from the multiple first keywords includes: selecting the at least one reference keyword from P first keywords that have the highest text miss frequency and hit the title of the electronic document, where P is a positive integer.
[0013] In some embodiments, determining the ease of search includes: respectively summing the miss frequencies of the multiple first keywords and the hit frequencies of the multiple second keywords to obtain a first summation result and a second summation result; calculating the sum of the first summation result and the second summation result; and determining the ease of search based on the ratio of the second summation result to the sum.
[0014] In some embodiments, the historical search data includes multiple search records, each search record includes multiple search sub-records, and calculating the miss frequencies of the multiple first keywords and the hit frequencies of the multiple second keywords includes: for each first keyword, determining the miss frequency of the first keyword based on the number of search sub-records including the first keyword; for each second keyword, determining the hit frequency of the second keyword based on the number of search sub-records including the second keyword.
[0015] In some embodiments, the historical search data includes multiple search records, each search record includes multiple search sub-records, the multiple search sub-records include successful search sub-records, the successful search sub-records include click sub-records, and the historical search data corresponding to the electronic document is the click sub-record in the successful sub-record, which includes the historical search data of the unique identifier of the electronic document.
[0016] According to a second aspect of the present disclosure, there is provided a data processing device, comprising: an acquisition module, configured to acquire historical search data corresponding to an electronic document, the historical search data comprising a plurality of search keywords; a determination module, configured to determine the searchability of the electronic document based on the historical search data, the searchability representing whether the electronic document is easy to be searched; a selection module, configured to select at least one reference keyword from the plurality of search keywords based on the searchability; and a recommendation module, configured to recommend the at least one reference keyword as a part of at least one of the title and the body of the electronic document.
[0017] According to a third aspect of the present disclosure, a data processing device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the data processing method described in any of the above embodiments based on instructions stored in the memory.
[0018] According to a fourth aspect of the present disclosure, a computer storable medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the data processing method described in any of the above embodiments is implemented.
[0019] In the above embodiments, the quality and accuracy of electronic documents are improved, thereby improving the efficiency of searching electronic documents. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0021] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
[0022] Figure 1is a flow chart illustrating a data processing method according to some embodiments of the present disclosure;
[0023] Figure 2 is a flow chart illustrating determining the searchability of an electronic document according to some embodiments of the present disclosure;
[0024] Figure 3 is a block diagram illustrating a data processing apparatus according to some embodiments of the present disclosure;
[0025] Figure 4 is a block diagram showing a data processing device according to some other embodiments of the present disclosure;
[0026] Figure 5 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0027] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.
[0028] At the same time, it should be understood that for the convenience of description, the sizes of the multiple parts shown in the drawings are not drawn according to the actual proportional relationship.
[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0030] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0031] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0032] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0033] Figure 1 is a flowchart illustrating a data processing method according to some embodiments of the present disclosure.
[0034] like Figure 1As shown, the data processing method includes: step S110, obtaining historical search data corresponding to the electronic document; step S120, determining the searchability of the electronic document; step S130, selecting at least one reference keyword; and step S140, recommending a reference keyword as a part of at least one of the title and text of the electronic document.
[0035] In step S110, historical search data corresponding to the electronic document is obtained. The historical search data includes a plurality of search keywords. For example, the electronic document is a knowledge document.
[0036] In some embodiments, when a search user searches a knowledge document library including multiple knowledge documents at a historical moment, the search engine generates historical search data and stores it in a search log system or database. For example, historical search data corresponding to an electronic document is obtained from the search log system or database.
[0037] In some embodiments, the historical search data includes multiple search records, each of which includes multiple search sub-records. For example, the multiple search sub-records include a successful search sub-record, the successful search sub-record includes a click sub-record, and the historical search data corresponding to the electronic document is the historical search data in which the click sub-record in the successful sub-record includes the unique identifier of the electronic document. In some embodiments, the unique identifier of the electronic document includes, but is not limited to, the title or ID (Identity Document) of the electronic document.
[0038] For example, the title of the knowledge document Doc[i] is KT[2]', i is a positive integer, and a search record corresponding to the knowledge document Doc[i] is represented by the following JSON (JavaScript Object Notation) string:
[0039] [{
[0040] "Number of searches": 1,
[0041] "Search sub-records":{
[0042] "Search keywords": [{KW[1]},{KW[2]},...,{KW[q]}],
[0043] "Return document title": [{KT[1]},{KT[2]},...,{KT[t]}],
[0044] "Title of the knowledge document last clicked by the customer service representative": "KT[t]",
[0045] }
[0046] },{
[0047] "Number of searches":2,
[0048] "Search sub-records":{
[0049] "Search keywords": [{KW[1]'},{KW[2]'},...,{KW[q']'}],
[0050] "Return document title": [{KT[1]'},{KT[2]'},...,{KT[t']'}],
[0051] "Title of the knowledge document last clicked by the customer service representative": "KT[2]'"
[0052] }
[0053] }].
[0054] In the above embodiment, a search record corresponding to the knowledge document Doc[i] includes two search sub-records, each of which is uniquely identified by the number of searches. Each search sub-record includes multiple search keywords, document titles returned based on the multiple search keywords, and the title of the knowledge document last clicked by the customer service representative.
[0055] In some embodiments, the search sub-records of the first search include search keywords KW[1], KW[2], ..., KW[q] and returned document titles KT[1], KT[2], ..., KT[t]. The search sub-records of the second search include search keywords KW[1]', KW[2]', ..., KW[q']' and returned document titles KT[1]', KT[2]', ..., KT[t']'. For example, the search keywords in each search sub-record are multiple search keywords obtained by the search engine by segmenting the search terms input by the customer service representative. The search engine returns the document title based on the multiple search keywords obtained by the segmentation. q, t, q', t' are all positive integers.
[0056] In some embodiments, a search record corresponds to a unique search user, and a search user corresponds to multiple search records. Each search record is when the search user performs at least one search for a customer by telephone, and each search generates a search sub-record. For example, the search user is a customer service representative.
[0057] The inventor believes that, according to the working characteristics of the customer service representative, the electronic document that the customer service representative reads for the last time during a telephone service is the final search result that the customer service representative wants and meets the customer service needs. That is to say, assuming that during a telephone service, the customer service representative conducted a total of Q searches, and in the result document of the Qth search, the customer service representative found the content he wanted to search for, but the result document of the first Q-1 searches did not contain the knowledge documents he needed, or even if there were the knowledge documents he needed, these documents were ranked at the back and were not seen by the customer service representative, so he had to search again. Q is a positive integer. For example, when Q>2, each search record in the historical search data of the present disclosure only stores two search sub-records, namely the Q-1th search sub-record as a failed search sub-record, and the Qth search sub-record as a successful search sub-record.
[0058] For example, in the above embodiment, the first search sub-record is a failed search sub-record, and the title of the knowledge document that the customer service representative last clicked is the click sub-record in the failed search sub-record. The second search sub-record is a successful search sub-record, and the title of the knowledge document that the customer service representative last clicked is the click sub-record in the successful search sub-record.
[0059] The above embodiment is only a simple example and does not represent search records in all cases. Those skilled in the art can reasonably set the storage form of the search record or the number of search sub-records in the search record according to some embodiments of the present disclosure.
[0060] For example, different search users U[1], ..., U[u] may perform multiple searches. Therefore, for the search information of each search user U[r] (u≥r≥1), JSON[r,1], JSON[r,2], ..., JSON[r,v] are used to represent the collected search records of the search user. r, u, v are all positive integers.
[0061] In step S120, the searchability of the electronic document is determined based on the historical search data. The searchability indicates whether the electronic document is easy to be searched.
[0062] Figure 2 is a flow chart illustrating determining the searchability of an electronic document according to some embodiments of the present disclosure.
[0063] like Figure 2 As shown, determining the searchability of an electronic document includes steps S121 to S123.
[0064] In step S121, multiple first keywords that do not hit the electronic document and multiple second keywords that hit the electronic document are determined from multiple search keywords. The first keyword is a search keyword that is not located in the title and text of the electronic document among the multiple search keywords. The second keyword is a search keyword that is located in at least one of the title and text of the electronic document among the multiple search keywords.
[0065] In step S122, the miss frequency of each first keyword and the hit frequency of each second keyword are calculated according to the historical search data.
[0066] In some embodiments, for each first keyword, the miss frequency of the first keyword is determined based on the number of search sub-records including the first keyword. For each second keyword, the hit frequency of the second keyword is determined based on the number of search sub-records including the second keyword.
[0067] For example, the number of search sub-records including the first keyword is determined as the miss frequency of the first keyword; and the number of search sub-records including the second keyword is determined as the hit frequency of the second keyword.
[0068] In step S123, the searchability is determined according to the miss frequencies of the first keywords and the hit frequencies of the second keywords.
[0069] In the above embodiment, the searchability is determined by the hit frequency and the miss frequency, and the correlation between the searchability and the user's search habits is improved, thereby further improving the quality and accuracy of the electronic document. The title or text of the electronic document is more closely matched with the search keywords, thereby improving the matching degree between the electronic document and the search keywords, and further improving the efficiency of searching for electronic documents.
[0070] In some embodiments, determining the searchability in step S123 is implemented as follows.
[0071] First, sum operations are performed on miss frequencies of multiple first keywords and hit frequencies of multiple second keywords respectively to obtain a first sum result and a second sum result.
[0072] Then, the sum of the first summation result and the second summation result is calculated.
[0073] Finally, the ease of search is determined according to the ratio of the second summation result to the total. For example, the ease of search is determined as the ratio of the second summation result to the total.
[0074] In some embodiments, the searchability includes the title searchability. The title searchability indicates whether the title of the electronic document is easy to search. The plurality of first keywords include a plurality of first title keywords that do not hit the title of the electronic document. The miss frequency includes the title miss frequency. The plurality of second keywords include a plurality of second title keywords that hit the title of the electronic document. The hit frequency includes the title miss frequency.
[0075] In some other embodiments, the searchability includes the text searchability. The text searchability indicates whether the text of the electronic document is easy to search. The plurality of first keywords include a plurality of first text keywords that do not hit the text of the electronic document. The miss frequency includes the text miss frequency. The plurality of second keywords include a plurality of second title keywords that hit the text of the electronic document. The hit frequency includes the text miss frequency.
[0076] In the case where the searchability is the title searchability or the text searchability, the title searchability or the text searchability may be determined in a manner or steps similar to those used in the above embodiments.
[0077] The process of determining the searchability of the title and the searchability of the text will be described in detail below with reference to a specific embodiment.
[0078] For the convenience of analysis, this embodiment introduces a search information node (hereinafter generally referred to as node). The fields of the search information node are as follows:
[0079] title_hit_keywords: an array consisting of two-tuples of the form (keyword, freq), where keyword is the second title keyword, freq is the frequency of the second title keyword (i.e., the number of searches with the first title keyword). title_hit_keywords is used to record the second title keywords and their frequencies that hit the title of the knowledge document;
[0080] title_missed_keywords: an array consisting of two-tuples of the form (keyword, freq), where keyword is the first title keyword, freq is the frequency of the first title keyword keyword (i.e., the number of times the first title keyword keyword is searched), and title_missed_keywords is used to record the first title keyword and its frequency of the title of the knowledge document that failed to hit.
[0081] text_hit_keywords: an array consisting of two-tuples of the form (keyword, freq), where keyword is the second text keyword, freq is the frequency of the second text keyword keyword (i.e., the number of searches using the second text keyword keyword), and text_hit_keywords is used to record the second text keywords and their frequencies that hit the text of the knowledge document.
[0082] text_missed_keywords: an array consisting of two-tuples of the form (keyword, freq), where keyword is the first text keyword, freq is the frequency of the first text keyword keyword (i.e., the number of times the first text keyword keyword is searched), and text_missed_keywords is used to record the first text keyword of the text that did not hit the knowledge document.
[0083] At the same time, this embodiment also introduces two other search information nodes, which are represented by map data structures: map<string,node> succ_data and map<string,node> fail_data. succ_data stores successful search information, its first element stores the title of the knowledge document, and its second element stores the second keyword (including the second title keyword and the second text keyword) and its frequency used by the search user when searching the knowledge document; fail_data stores failed search information, its first element stores the title of the knowledge document, and its second element stores the first keyword (including the first title keyword and the first text keyword) and its frequency used by the search user when searching the knowledge document.
[0084] In some embodiments, succ_data and fail_data each include multiple child nodes, the first element of each child node is the title of the knowledge document, and the second element is the node node defined above.
[0085] For example, the knowledge documents in a knowledge management system or knowledge document library are Doc[1], Doc[2], ..., Doc[n], where for each knowledge document Doc[i] (n≥i≥1), Doc[i].title represents the title of Doc[i], and Doc[i].text represents the text of Doc[i]. n is a positive integer.
[0086] For each search user U[r] (u≥i≥1), the search user performs v searches on the knowledge documents Doc[1], Doc[2], ..., Doc[n] in the knowledge document library, and the search records are stored in JSON[r,1], JSON[r,2], ..., JSON[r,v] respectively.
[0087] For example, for each search record JSON[r,j](v≥j≥1) of each search user U[r](u≥r≥1), the maximum number of searches in JSON[r,j] is 2. According to the search habits of customer service representatives, the last search sub-record is the sub-record for successful search. j is a positive integer.
[0088] If the value of the knowledge document title last clicked by the customer service representative in the second search sub-record in JSON[r,j] is not in succ_data, perform the following operations:
[0089] First, create a new node node_a. In the initial state, the value of succ_data is empty;
[0090] Then, for each KW[a']' (q'≥a'≥1) in the search keyword in the second search sub-record in JSON[r,j], if KW[a']' exists in the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then (KW[a']', 1) is added to the title_hit_keywords of the node_a, and 1 is the frequency freq_a'; otherwise, (KW[a']', 1) is added to the node_a. e_a node’s title_missed_keywords; if KW[a']' exists in the body of the knowledge document corresponding to the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then add (KW[a']', 1) to node_a’s text_hit_keywords; otherwise, add (KW[a']', 1) to node_a’s text_missed_keywords;
[0091] Finally, the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in (JSON[r,j], node_a) is stored in succ_data.
[0092] If the value of the knowledge document title last clicked by the customer service representative in the second search sub-record in JSON[r,j] is in succ_data (that is, if the value of the knowledge document title last clicked by the customer service representative in the second search sub-record in JSON[r,j] already exists in succ_data), perform the following operations:
[0093] First, find the record whose first element is equal to the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j] from succ_data, record it as data_a, and set the second element of data_a as node_a;
[0094] Then, for each KW[a']' (q'≥a'≥1) in the search keyword in the second search sub-record in JSON[r,j], if KW[a']' exists in the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then if the title_hit_keywords of node_a exists in a certain tuple (KW[a']',freq_a'), then freq_a'=freq_a'+1; otherwise, add (KW[a']',1) to node_a. de_a's title_hit_keywords; if KW[a']' exists in the text corresponding to the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then if node_a's text_hit_keywords exists in a tuple (KW[a']',freq_a'), then freq_a'=freq_a'+1; otherwise, add (KW[a']',1) to node_a's text_hit_keywords;
[0095] Finally, the modified node_a is put back into data_a, and the modified data_a is put back into succ_data.
[0096] If the value of the knowledge document title last clicked by the customer service representative in the second search sub-record in JSON[r,j] is not in fail_data, then perform the following operations:
[0097] First, create a new node node_b. In the initial state, the value of fail_data is empty.
[0098] Then, for each KW[a] (q≥a≥1) in the search keyword in the first search sub-record in JSON[r,j], if KW[a] exists in the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then (KW[a], 1) is added to the title_hit_keywords of the node_b, where 1 is the frequency freq_a; otherwise, (KW[a], 1) is added to the title_missed_keywords of the node_b; if KW[a] exists in the text of the knowledge document corresponding to the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then (KW[a], 1) is added to the text_hit_keywords of the node_b; otherwise, (KW[a], 1) is added to the text_missed_keywords of the node_b;
[0099] Finally, the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in (JSON[r,j], node_b) is stored in fail_data.
[0100] If the value of the knowledge document title last clicked by the customer service representative in the second search sub-record in JSON[r,j] is in fail_data (that is, if the value of the knowledge document title last clicked by the customer service representative in the second search sub-record in JSON[r,j] already exists in fail_data), perform the following operations:
[0101] First, find the record whose first element is equal to the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j] from fail_data, record it as data_b, and set the second element of data_b as node_b;
[0102] Then, for each KW[a] (q≥a≥1) in the search keyword in the first search sub-record in JSON[r,j]: if KW[a] exists in the value of the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then if the title_hit_keywords of node_b exists in a certain tuple (KW[a],freq_a), then freq_a=freq_a+1; otherwise, add (KW[a],1) to the title_hit_keywords of node_b. If KW[a] exists in the body of the knowledge document corresponding to the title of the knowledge document last clicked by the customer service representative in the second search sub-record in JSON[r,j], then if the text_hit_keywords of node_b exists in a tuple (KW[a],freq_a), then freq_a=freq_a+1; otherwise, add (KW[a],1) to the text_hit_keywords of node_b;
[0103] Finally, the modified node_b is put back into data_b, and the modified data_b is put back into fail_data.
[0104] In some embodiments, for a user whose maximum number of searches is 1, it is determined whether the value of the knowledge document title last clicked by the customer service representative in the first search sub-record in JSON[r,j] is in succ_data, and similar operations as those for a user whose maximum number of searches is 2 are performed based on different determination results.
[0105] In some embodiments, after performing similar operations as described above on all search records of all users, for any knowledge document Doc[i] (n≥i≥1), the title searchability (ETSBT, Ease To Seach By Title) and text searchability (ETSBX, Ease To Seach By teXt) of the knowledge document are calculated in the following ways.
[0106] In some embodiments, for any knowledge document Doc[i] (n≥i≥1), the title search ease (ETSBT) of the knowledge document is calculated as follows.
[0107] For the child node whose first element in succ_data is Doc[i].title, let the second element of the child node be node_a. Similarly, for the child node whose first element in fail_data is the title Doc[i].title of the knowledge document Doc[i], let the second element of the child node be node_b.
[0108] For the knowledge document Doc[i], the total title hit frequency title_hit_cnt of the multiple second title keywords in the multiple search keywords is initialized to 0, and the total title miss frequency tilte_missed_cnt of the multiple first title keywords in the multiple search keywords is initialized to 0. The total title hit frequency is the sum of the title hit frequencies of the multiple second title keywords, which is the second summation result in the above embodiment. The total title miss frequency is the sum of the title miss frequencies of the multiple first title keywords, which is the first summation result in the above embodiment.
[0109] For each tuple (KW[a']', freq_a') in title_hit_keywords in node_a, title_hit_cnt = title_hit_cnt + freq_a'.
[0110] For each bigram (KW[a], freq_a) in title_hit_keywords in node_b, title_hit_cnt=title_hit_cnt+freq_a.
[0111] For each tuple (KW[a']', freq_a') in title_missed_keywords in node_a, title_missed_cnt = title_missed_cnt + freq_a'.
[0112] For each tuple (KW[a], freq_a) in title_hit_keywords of node_b, title_missed_cnt = title_missed_cnt + freq_a.
[0113] Through the above calculation, the total title hit frequency title_hit_cnt of multiple second title keywords and the total title miss frequency tilte_missed_cnt of multiple first title keywords of the knowledge document Doc[i] can be determined.
[0114] Based on the total title hit frequency title_hit_cnt of multiple second title keywords and the total title miss frequency tilte_missed_cnt of multiple first title keywords of the knowledge document Doc[i], the title searchability ETSBT(Doc[i]) of the knowledge document Doc[i] can be determined.
[0115] The calculation formula of title searchability ETSBT(Doc[i]) of knowledge document Doc[i] is:
[0116]
[0117] The larger the value of ETSBT(Doc[i]), the easier it is to search for the title Doc[i].title of the knowledge document Doc[i].
[0118] In some embodiments, for any knowledge document Doc[i] (n≥i≥1), the text ease of search (ETSBX, Ease To Seach By teXt) of the knowledge document is calculated in the following manner.
[0119] For the child node whose first element of succ_data is Doc[i].title, let the second element of the child node be node_a. Similarly, for the child node whose first element of fail_data is Doc[i].title, let the second element of the child node be node_b.
[0120] For the knowledge document Doc[i], the total text hit frequency text_hit_cnt of the multiple second text keywords in the multiple search keywords is initialized to 0, and the total text miss frequency text_missed_cnt of the multiple first text keywords in the multiple search keywords is initialized to 0. The total text hit frequency is the sum of the text hit frequencies of the multiple second text keywords, which is the second summation result in the above embodiment. The total text miss frequency is the sum of the text miss frequencies of the multiple first text keywords, which is the first summation result in the above embodiment.
[0121] For each tuple (KW[a']',freq_a') in text_hit_keywords in node_a, text_hit_cnt=text_hit_cnt+freq_a'.
[0122] For each bigram (KW[a],freq_a) in text_hit_keywords in node_b, text_hit_cnt=text_hit_cnt+freq_a.
[0123] For each tuple (KW[a']',freq_a') in text_missed_keywords in node_a, text_missed_cnt=text_missed_cnt+freq_a'.
[0124] For each bigram (KW[a],freq_a) in text_hit_keywords in node_b, text_missed_cnt=text_missed_cnt+freq_a.
[0125] Through the above calculation, the text hit frequency text_hit_cnt and the text miss frequency text_missed_cnt of multiple search keywords of the knowledge document Doc[i] can be determined.
[0126] According to the title hit frequency text_hit_cnt and title miss frequency text_missed_cnt of multiple search keywords of the knowledge document Doc[i], the title searchability ETSBT(Doc[i]) of the knowledge document Doc[i] can be determined.
[0127] The calculation formula of title searchability ETSBT(Doc[i]) of knowledge document Doc[i] is:
[0128]
[0129] The larger the value of ETSBX(Doc[i]), the easier it is to search for the main text Doc[i].text of the knowledge document Doc[i].
[0130] It should be understood that the above embodiment is only an example of determining the searchability of a title and a body text, and those skilled in the art may use different data structures to record search information according to different requirements, and may even use different process sequences to determine the searchability of a title and a body text.
[0131] return Figure 1 After determining the searchability, execute step S130.
[0132] In step S130, at least one reference keyword is selected from a plurality of search keywords according to the searchability.
[0133] For example, step S130 is implemented in the following manner.
[0134] First, when the searchability is less than or equal to the threshold, multiple first keywords of the electronic documents that have not been hit in the multiple search keywords are obtained. For example, Figure 2For example, the multiple first keywords determined in step S121 may also be executed. Figure 2 The step S121 shown is similar to the step of determining a plurality of first keywords that do not hit the electronic document from the plurality of search keywords.
[0135] Then, the miss frequencies of multiple first keywords are obtained. For example, Figure 2 The miss frequency of each first keyword calculated in step S122. Figure 2 A similar step to step S122 is shown, in which the miss frequency of each first keyword is calculated based on the historical search data.
[0136] Finally, at least one reference keyword is selected from the multiple first keywords according to the ranking results of the miss frequencies of the multiple first keywords.
[0137] In the above embodiment, by selecting at least one reference keyword from multiple first keywords based on the ranking results of the miss frequencies of multiple first keywords when the searchability is less than or equal to the threshold, the accuracy of selecting the reference keywords is further improved, the quality and accuracy of the electronic document is further improved, and the efficiency of searching the electronic document is further improved.
[0138] In some embodiments, at least one reference keyword is selected from the N first keywords with the highest unhit frequency, where N is a positive integer.
[0139] In the case where the searchability is the title searchability, the threshold includes a title threshold. In some embodiments, at least one reference keyword is selected from the M first keywords with the highest title miss frequency and that hit the body of the electronic document. M is a positive integer. In some embodiments, the title threshold is 0.1 and M is 5. Those skilled in the art can reasonably set the title threshold and M as needed.
[0140] For example, for the child node whose first element in fail_data is Doc[i].title, let the second element of the child node be node_b. Recommend the five search keywords with the highest frequency in title_missed_keywords in node_b but not appearing in text_hit_keywords of node_b to the editor of the knowledge document as a reference for modifying Doc[i].title.
[0141] In the case where the searchability is the text searchability, the threshold includes the text threshold. In some embodiments, at least one reference keyword is selected from the P first keywords with the highest text miss frequency and that hit the title of the electronic document. P is a positive integer. In some embodiments, the text threshold is 0.1 and N is 10. Those skilled in the art can reasonably set the text threshold and N as needed.
[0142] For example, for the child node whose first element of fail_data is Doc[i].title, let the second element of the child node be node_b. Recommend the 10 search keywords with the highest frequency in text_missed_keywords of node_b but not appearing in title_hit_keywords of node_b to the editor of the knowledge document as a reference for modifying Doc[i].text.
[0143] In step S140, at least one reference keyword is recommended as a part of at least one of the title and the body of the electronic document. For example, at least one reference keyword is recommended to the editor of the knowledge document as a basis for the editor to modify the title and the body of the knowledge document.
[0144] In the above embodiment, by selecting at least one reference keyword from multiple search keywords according to the ease of search, and recommending the at least one search keyword as a part of the title or body of the electronic document, the quality and accuracy of the electronic document are improved, so that the title or body of the electronic document is more matched with the search keyword, the matching degree between the electronic document and the search keyword is improved, and the efficiency of the search engine in searching the electronic document is further improved. The above embodiment also indirectly improves the search experience of the search user.
[0145] Figure 3 is a block diagram illustrating a data processing apparatus according to some embodiments of the present disclosure.
[0146] like Figure 3 As shown, the data processing device 3 includes an acquisition module 31 , a determination module 32 , a selection module 33 and a recommendation module 34 .
[0147] The acquisition module 31 is configured to acquire the historical search data corresponding to the electronic document, for example, by performing the following steps: Figure 1 Step S110 is shown. The historical search data includes a plurality of search keywords.
[0148] The determination module 32 is configured to determine the searchability of the electronic document based on the historical search data, for example, by performing the following steps: Figure 1 Step S120 is shown. The searchability indicates whether the electronic document is easy to be searched.
[0149] The selection module 33 is configured to select at least one reference keyword from a plurality of search keywords according to the ease of search, for example, by performing the following steps: Figure 1 Step S130 is shown.
[0150] The recommendation module 34 is configured to recommend at least one reference keyword as a part of at least one of the title and the body of the electronic document, for example, by performing the following steps: Figure 1 Step S140 is shown.
[0151] Figure 4 is a block diagram showing a data processing device according to some other embodiments of the present disclosure.
[0152] like Figure 4 As shown, the data processing device 4 includes a memory 41; and a processor 42 coupled to the memory 41. The memory 41 is used to store instructions for executing the corresponding embodiments of the data processing method. The processor 42 is configured to execute the data processing method in any of the embodiments of the present disclosure based on the instructions stored in the memory 41. For example, the data processing device 4 and Figure 3 The data processing device 3 has the same or similar function and structure.
[0153] Figure 5 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.
[0154] like Figure 5 As shown, the computer system 50 may be embodied in the form of a general purpose computing device. The computer system 50 includes a memory 510, a processor 520, and a bus 500 that connects the various system components.
[0155] The memory 510 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions for executing at least one of the corresponding embodiments of the data processing method. The non-volatile storage medium may include, but is not limited to, a disk memory, an optical memory, a flash memory, etc.
[0156] The processor 520 can be implemented by a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, etc. Discrete hardware components. Accordingly, each module such as the judgment module and the determination module can be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or can be implemented by a dedicated circuit that executes corresponding steps.
[0157] The bus 500 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.
[0158] The computer system 50 may also include an input / output interface 530, a network interface 540, a storage interface 550, etc. These interfaces 530, 540, 550, the memory 510, and the processor 520 may be connected via a bus 500. The input / output interface 530 may provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 540 may provide a connection interface for various networked devices. The storage interface 550 may provide a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.
[0159] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.
[0160] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.
[0161] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.
[0162] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.
[0163] The data processing method and device, and the computer storable medium in the above-mentioned embodiments improve the quality and accuracy of electronic documents, thereby improving the efficiency of searching electronic documents.
[0164] So far, the data processing method and device, and the computer storable medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.
Claims
1. A data processing method, include: Acquire historical search data corresponding to the electronic document, wherein the historical search data includes a plurality of search keywords; Determining the searchability of the electronic document according to the historical search data includes: determining, from the multiple search keywords, a plurality of first keywords that do not hit the electronic document and a plurality of second keywords that hit the electronic document, respectively, wherein the first keywords are search keywords that are not located in the title and text of the electronic document among the multiple search keywords, and the second keywords are search keywords that are located in at least one of the title and text of the electronic document among the multiple search keywords; calculating, according to the historical search data, the miss frequency of each first keyword and the hit frequency of each second keyword; determining the searchability according to the miss frequencies of the multiple first keywords and the hit frequencies of the multiple second keywords, wherein the searchability indicates whether the electronic document is easy to be searched; According to the searchability, selecting at least one reference keyword from the multiple search keywords includes: when the searchability is less than or equal to a threshold, obtaining multiple first keywords from the multiple search keywords that do not hit the electronic document; obtaining the miss frequencies of the multiple first keywords; and selecting at least one reference keyword from the multiple first keywords according to the ranking results of the miss frequencies of the multiple first keywords; The at least one reference keyword is recommended as a part of at least one of a title and a body of the electronic document.
2. The data processing method according to claim 1, in, Selecting at least one reference keyword from the plurality of first keywords comprises: The at least one reference keyword is selected from the N first keywords with the highest unhit frequencies, where N is a positive integer.
3. The data processing method according to claim 1, in, The searchability includes title searchability, the title searchability indicates whether the title of the electronic document is easy to be searched, the threshold includes a title threshold, the multiple first keywords include multiple first title keywords that do not hit the title of the electronic document, and the miss frequency includes a title miss frequency.
4. The data processing method according to claim 3, in, Selecting at least one reference keyword from the plurality of first keywords comprises: The at least one reference keyword is selected from the M first keywords that have the highest title miss frequency and hit the body of the electronic document, where M is a positive integer.
5. The data processing method according to claim 1, in, The searchability includes the text searchability, the text searchability indicates whether the text of the electronic document is easy to search, the threshold includes the text threshold, the multiple first keywords include multiple first text keywords that do not hit the text of the electronic document, and the miss frequency includes the text miss frequency.
6. The data processing method according to claim 5, in, Selecting at least one reference keyword from the plurality of first keywords comprises: The at least one reference keyword is selected from P first keywords that have the highest text miss frequency and hit the title of the electronic document, where P is a positive integer.
7. The data processing method according to claim 1, in, Determining the searchability includes: respectively summing up the miss frequencies of the plurality of first keywords and the hit frequencies of the plurality of second keywords to obtain a first summation result and a second summation result; Calculating the sum of the first summation result and the second summation result; The searchability is determined according to a ratio of the second summation result to the total.
8. The data processing method according to claim 1, in, The historical search data includes a plurality of search records, each search record includes a plurality of search sub-records, and calculating the miss frequencies of the plurality of first keywords and the hit frequencies of the plurality of second keywords includes: For each first keyword, determining the miss frequency of the first keyword according to the number of search sub-records including the first keyword; For each second keyword, the hit frequency of the second keyword is determined according to the number of search sub-records including the second keyword.
9. The data processing method according to claim 1, in, The historical search data includes multiple search records, each search record includes multiple search sub-records, the multiple search sub-records include successful search sub-records, the successful search sub-records include click sub-records, and the historical search data corresponding to the electronic document is the click sub-record in the successful sub-record, which includes the historical search data of the unique identifier of the electronic document.
10. A data processing device, include: An acquisition module, configured to acquire historical search data corresponding to the electronic document, wherein the historical search data includes a plurality of search keywords; The determination module is configured to determine the searchability of the electronic document according to the historical search data, including: determining, from the multiple search keywords, a plurality of first keywords that do not hit the electronic document and a plurality of second keywords that hit the electronic document, respectively, wherein the first keywords are search keywords that are not located in the title and body of the electronic document among the multiple search keywords, and the second keywords are search keywords that are located in at least one of the title and body of the electronic document among the multiple search keywords; respectively calculating the miss frequency of each first keyword and the hit frequency of each second keyword according to the historical search data; determining the searchability according to the miss frequency of the multiple first keywords and the hit frequency of the multiple second keywords, wherein the searchability indicates whether the electronic document is easy to be searched; The selection module is configured to select at least one reference keyword from the multiple search keywords according to the searchability, including: when the searchability is less than or equal to a threshold, obtaining multiple first keywords that do not hit the electronic document from the multiple search keywords; obtaining the miss frequencies of the multiple first keywords; and selecting at least one reference keyword from the multiple first keywords according to the ranking results of the miss frequencies of the multiple first keywords; The recommendation module is configured to recommend the at least one reference keyword as a part of at least one of a title and a body of the electronic document.
11. A data processing device, include: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the data processing method according to any one of claims 1 to 9 based on instructions stored in the memory.
12. A computer storable medium having computer program instructions stored thereon, which, when executed by a processor, implement the data processing method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Method for sorting search results by adding labels into search results
CN101963966A
Search suggestion providing method and apparatus
CN104933081A
Method, equipment and electronic equipment for content document sorting
CN105653737A
Method and system for calculating retrieval precision of internet keyword
CN106560808A
Document processing method and device
CN107818091A
