Intelligent input recommendation method and system based on multi-source corpus and parallel index

Abstract

The invention provides an intelligent input recommendation method and system based on multi-source corpora and parallel indexing, and belongs to the technical field of intelligent input recommende.The method comprises the steps that core corpora are obtained, and weighted word frequency calculation is conducted on the core corpora to obtain final word frequency; extracting a plurality of first character prefixes of all vocabularies in the core corpus, and grouping according to the length of the character prefixes to obtain a plurality of prefix groups; distributing an independent thread for each prefix group and constructing index mapping in the independent thread; obtaining input associative query content issued by a user, and performing degradation matching according to the independent thread based on the input associative query content to obtain a matching result; according to the method, the core pain point of the association function of the existing input method can be solved, and the method has the characteristics of reproducibility, easiness in landing and wide adaptation, and is suitable for multi-scene input requirements of office, vehicle-mounted, embedded and the like.

Description

Technical Field

[0001] This invention belongs to the technical field of intelligent input recommendation, specifically relating to an intelligent input recommendation method and system based on multi-source corpus and parallel indexing. Background Technology

[0002] In the diverse application scenarios of Chinese input methods, different user groups (such as office workers, embedded device users, and professionals) have placed higher demands on the efficiency, accuracy, and lightweight adaptability of the predictive text function. However, current input method predictive text technology still has many core pain points that are out of touch with actual needs, specifically reflected in the following typical scenarios: 1. Insufficient accuracy of professional terminology association in office scenarios: When office workers input industry-specific terms (such as "report formula" in the financial field and "code debugging" in the IT field), existing input methods mostly rely on general corpora for association, without differentiating the scenario attributes of the corpus. For example, when inputting "report", the system often mixes in everyday words such as "report news" and "report template", while professional terms with high-frequency office needs such as "report export" and "report verification" are ranked lower. Users need to search multiple times to select the target word, which increases the time for a single professional term input by 2-3 times compared to the ideal state, directly reducing the operational efficiency of office documents, data entry and other scenarios.

[0003] 2. The contradiction between efficiency and adaptability in dictionary building for embedded devices (vehicle terminals, smartwatches): Embedded devices are limited by hardware computing power (usually CPU ≤ 2 cores, memory ≤ 1GB). Existing input methods use single-threaded processing of multi-source corpora (such as general basic corpora + device-specific corpora). When building the associative dictionary, the initialization time for millions of texts can be as long as 4-5 hours, causing the input method to be in a "loading" state for a long time when the device is first started, and users cannot use it immediately. If the corpus is simplified in order to shorten the loading time, it will result in insufficient coverage of associative words - for example, when inputting "navigation" in a vehicle scenario, it will not be able to associate device-specific words such as "navigation destination" and "navigation voice switching", thus losing the value of scenario adaptation.

[0004] 3. Poor association fault tolerance for common user short words and incomplete inputs: In daily input, common users often input short words (e.g., "report" and "division"), incomplete words (e.g., "report division" instead of "report analysis"), or rare words (e.g., industry abbreviations "ERP" and "KPI"). Existing association technologies mostly use "single prefix matching" logic. If there is no corresponding index for the input word, the system returns an empty result, and the user needs to manually input the complete word. This is particularly true for older users and people who are not familiar with typing. In these cases, the empty result accounts for 15-20% of daily input, increasing the complexity of operation and causing resistance to use, which reduces the stickiness of the input method.

[0005] From the existing technical solutions, the core implementation logic of the current input method association function still has significant defects, which cannot cover the above-mentioned scene requirements. The specific manifestations are as follows: 1. Dictionary construction: single-thread processing + no differentiated corpus fusion, low efficiency and accuracy: mainstream technologies use single-thread reading and splicing of multi-source corpora (general corpora, scene corpora, and user historical corpora). Without weighting according to the scene value of the corpus, "general corpora squeeze professional corpora", and the accuracy of professional scene association is insufficient. At the same time, the single-thread processing mode is not optimized for the algorithmic limit of embedded devices. The initialization time of a million-level corpus far exceeds the user's acceptable range (usually the user tolerates ≤5 minutes of loading time), forming a contradiction between efficiency and accuracy.

[0006] 2. Matching mechanism: single dimension + low fault tolerance design, insufficient adaptability: existing solutions support prefix matching, but only use a two-level degradation logic of "complete word → single character", lacking an intermediate transition link such as "long prefix completion" (e.g., input "report division" and intercept "report" to match). When inputting incomplete words, it is easy to get an empty result. Some technologies try to introduce deep learning models to improve semantic relevance, but the model memory occupancy ≥200MB, which cannot adapt to the hardware limit of embedded devices, and relies on cloud computing power support, which is completely ineffective in offline scenarios, limiting the scope of application.

[0007] 3. Index structure: flat design leads to low query efficiency: Most input methods use a "prefix - word frequency" flat key-value structure. When querying, the entire prefix corresponding to the word list needs to be traversed. In the case of a million-level dictionary, the single association word query delay is more than 100ms. In high-frequency input scenarios such as office typing and instant messaging (user typing speed ≥80 words / minute), this delay will cause "association words lagging behind input rhythm" lag phenomenon, disrupting the smoothness of input.

[0008] In summary, the existing input method association technology cannot simultaneously meet the comprehensive needs of "high efficiency construction, high precision association, lightweight adaptation, and high fault tolerance", especially in core scenarios such as professional office, embedded devices, and short word input of ordinary users, the contradictions between technical defects and actual user needs are increasingly prominent, and an intelligent input recommendation method that can cooperatively solve the above-mentioned pain points is urgently needed. SUMMARY

[0009] To solve the above technical problems, the present application provides an intelligent input recommendation method and system based on multi-source corpus and parallel index to solve the problems in the background art.

[0010] In a first aspect, the present application provides the following technical solution, an intelligent input recommendation method based on multi-source corpus and parallel index, comprising: Obtaining core corpus, calculating weighted word frequency of the core corpus to obtain final word frequency; Extracting the first several character prefixes of all words in the core corpus and grouping them according to the length of the character prefixes to obtain several prefix groups; Assigning an independent thread to each of the prefix groups and constructing an index mapping in the independent thread; Obtaining input association query content issued by a user, performing degradation matching based on the input association query content according to the independent thread to obtain a matching result; Performing three-level index query based on the matching result and the index mapping to output a recommendation result.

[0011] Compared with the prior art, the present application has the following advantages: the present application solves the problems of "memory occupancy > 200MB" and "dependence on cloud computing power" of the existing deep learning model, ensures the stable operation of the scheme on embedded devices (vehicle terminal, smart watch) with CPU ≤ 2 cores and memory ≤ 1GB, and at the same time, the present application discards the traditional semantic model (such as BERT, Word2Vec), and only realizes precise association through "weighted word frequency + scene label", avoiding model loading and occupying memory; the present application only retains words with a word frequency ≥ 1.0 (eliminates extremely low frequency and rare words), further compresses the index file size (from 50MB to 35MB), the present application loads on demand when the user first queries, reduces the memory occupancy at startup (startup memory ≤ 30MB), the present application caches the prefix index recently queried by the user to the memory, and directly reuses the subsequent query to avoid repeated file reading, the present application forms a complete technical chain of "corpus fusion → high efficiency construction → fast query → fault tolerance matching → lightweight adaptation" through the synergistic effect of the above improvements, which well solves the core pain points of the existing input method association function, and the scheme has the characteristics of "reproducible, easy to land, and wide adaptation", and is suitable for office, vehicle, embedded and other multi-scene input needs.

[0012] Preferably, the step of performing weighted word frequency calculation on the core corpus to obtain the final word frequency comprises: loading a general basic corpus, a scene professional corpus, and a user history corpus, performing text information on the general basic corpus, the scene professional corpus, and the user history corpus respectively, removing data noise to obtain a general basic de-noised corpus, a scene professional de-noised corpus, and a user history de-noised corpus, identifying the basic word frequency, the scene word frequency, and the history word frequency of each word in the general basic de-noised corpus, the scene professional de-noised corpus, and the user history de-noised corpus respectively, calculating the final word frequency of each word based on the basic word frequency , the scene word frequency , and the history word frequency . : ; wherein, are the first weight, the second weight, and the third weight respectively, .

[0013] Preferably, the step of extracting the first several character prefixes of all words in the core corpus and grouping them according to the length of the character prefixes to obtain several prefix groups comprises: extracting the first character prefix, the second character prefix, and so on until the Nth character prefix of all words in the core corpus in sequence, and storing the character prefixes belonging to the same character length in the same prefix group to obtain N prefix groups.

[0014] Preferably, the step of assigning an independent thread to each of the prefix groups and constructing an index mapping in the independent thread comprises: assigning an independent thread to each of the prefix groups of different character lengths, in each of the independent threads, traversing all words starting with the character prefix in the core corpus for each character prefix in each of the prefix groups, and taking the traversed words as the associated words of the character prefix, traversing and searching the associated words in the preset scene keyword library for any complete keyword, if a corresponding complete keyword is searched in the preset scene keyword library, assigning the corresponding scene label to the associated words, if multiple corresponding complete keywords are searched in the preset scene keyword library, preferentially assigning the scene label of the source corpus to the associated words, if no corresponding complete keyword is searched in the preset scene keyword library, assigning a general scene label to the associated words, and integrating the prefix traversal, keyword extraction, and scene label assignment processes in the independent thread to obtain an index mapping.

[0015] Preferably, the step of performing descending matching based on the input association query content according to the independent threads to obtain a matching result comprises: extracting the first N characters of the association query content as input content, performing retrieval in the independent thread corresponding to the Nth prefix group, and if a corresponding character prefix is retrieved, storing the output in the matching result; if a corresponding character prefix is not retrieved, extracting the first N-1 characters of the association query content as input content, performing retrieval in the independent thread corresponding to the N-1th prefix group, and if a corresponding character prefix is retrieved, storing the output in the matching result; if a corresponding character prefix is not retrieved, extracting the first N-2 characters of the association query content as input content, performing retrieval in the independent thread corresponding to the N-2th prefix group, and repeating the descending matching process until the first character of the association query content is extracted as input content, retrieval is performed in the independent thread corresponding to the 1st prefix group, and if a corresponding character prefix is retrieved, the output is stored in the matching result; if a corresponding character prefix is not retrieved, extracting the last 1 character of the association query content as input content, performing retrieval in the independent thread corresponding to the 1st prefix group, and if a corresponding character prefix is retrieved, the output is stored in the matching result.

[0016] Preferably, the step of performing three-level index query based on the matching result and the index mapping to output a recommended result comprises: extracting a corresponding character prefix from the matching result, extracting a corresponding associated vocabulary corresponding to the character prefix from the index mapping, eliminating irrelevant vocabulary in the associated vocabulary according to a scene label to obtain a filtered vocabulary, performing deduplication processing on the filtered vocabulary and arranging the filtered vocabulary in descending order according to a corresponding final word frequency, recommending the first several filtered vocabularies in the sorted order to obtain a recommended result.

[0017] In a second aspect, the present application provides the following technical solution: an intelligent input recommendation system based on multi-source corpus and parallel index, the system comprising: a calculation module configured to obtain a core corpus, and perform weighted word frequency calculation on the core corpus to obtain a final word frequency; a grouping module configured to extract the first several character prefixes of all vocabularies in the core corpus and group the character prefixes according to the length of the character prefixes to obtain several prefix groups; a mapping module configured to assign an independent thread to each of the prefix groups and construct an index mapping in the independent thread; a matching module configured to obtain input association query content issued by a user, and perform descending matching based on the input association query content according to the independent threads to obtain a matching result. a recommendation module configured to perform a three-level index query based on the matching result and the index mapping to output a recommendation result.

[0018] Preferably, the calculation module is configured to: load a general basic corpus, a scene professional corpus and a user history corpus, perform text information on the general basic corpus, the scene professional corpus and the user history corpus respectively, remove data noise to obtain a general basic de-noised corpus, a scene professional de-noised corpus and a user history de-noised corpus, identify a basic word frequency, a scene word frequency and a history word frequency of each word in the general basic de-noised corpus, the scene professional de-noised corpus and the user history de-noised corpus respectively, calculate a final word frequency of each word based on the basic word frequency , the scene word frequency and the history word frequency . : ; wherein are a first weight, a second weight and a third weight respectively, .

[0019] In a third aspect, the present application provides the following technical solution, a computer comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the computer program when implementing the intelligent input recommendation method based on multi-source corpora and parallel indexes as described above.

[0020] In a fourth aspect, the present application provides the following technical solution, a storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the intelligent input recommendation method based on multi-source corpora and parallel indexes as described above. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0022] Figure 1 a flowchart of the intelligent input recommendation method based on multi-source corpora and parallel indexes provided by the first embodiment of the present application; Figure 2 a structural block diagram of the intelligent input recommendation system based on multi-source corpora and parallel indexes provided by the second embodiment of the present application; Figure 3The schematic diagram of the hardware structure of the computer provided for another embodiment of the present application is shown.

[0023] The present application will be further described below in conjunction with the drawings and the description of the drawings. DETAILED DESCRIPTION

[0024] Embodiments of the present application are described in detail below with reference to examples shown in the drawings, in which the same or similar notations are used to denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the embodiments of the present application, and cannot be understood as limiting the present application.

[0025] In the description of the embodiments of the present application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0026] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly specified and limited.

[0027] In the embodiments of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection", "fixing" and the like should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0028] Embodiment one In one embodiment of the present application, as shown in Figure 1 A smart input recommendation method based on multi-source corpus and parallel index, comprising: S1, obtaining a core corpus, performing weighted word frequency calculation on the core corpus to obtain a final word frequency; The step S1 is specifically: Load general basic corpus, scenario-specific corpus, and user history corpus. Perform text information processing on the general basic corpus, scenario-specific corpus, and user history corpus respectively to remove data noise, resulting in general basic denoised corpus, scenario-specific denoised corpus, and user history denoised corpus. Identify the basic word frequency, scenario word frequency, and historical word frequency of each word in the general basic denoised corpus, scenario-specific denoised corpus, and user history denoised corpus, and base the results on the basic word frequency. The frequency of the scene The historical word frequency Calculate the final word frequency for each word. : ; In the formula, They are the first weight, the second weight, and the third weight, respectively. ; The general basic corpus comes from the THUOCL general vocabulary database, which covers high-frequency everyday words and has a weight coefficient of 0.3 (to meet basic input needs without squeezing out the priority of professional vocabulary). The scenario-specific corpus comes from texts in vertical fields such as office, in-vehicle, and medical, such as the "ck.txt" office corpus, which contains domain-specific terms and has a weight coefficient of 0.5 (to prioritize matching in professional scenarios and solve the problem of associating "report" with "report formula" instead of "report news"). The user history corpus comes from the user's historical input records, which reflects personal word usage habits and has a weight coefficient of 0.2 (to adapt to personalized needs, such as prioritizing "function definition" when a programmer inputs "function"). Therefore, the first, second, and third weights mentioned above are set to 0.3, 0.5, and 0.2, respectively. It should be noted that if a user is using the service for the first time or has less than 10 historical input entries, a dynamic weight compensation mechanism is designed. In this case, the third weight of the user's historical corpus will be allocated to the two types of corpus according to the original weight ratio (0.3:0.5) of the general basic corpus and the scenario-specific corpus. This ensures that even when there is a lack of user historical data, the first weight is 0.375 and the second weight is 0.625. The accuracy of association in professional scenarios can still be maintained by increasing the actual weight ratio of the scenario-specific corpus (from 0.5 to 0.625). Once the user's historical corpus records 10 or more entries, the basic weight configuration of 0.2 will be automatically restored, gradually enhancing the personalized association capability. For the calculation of the final word frequency, for example, the basic word frequency of the word "report generation" is 3, the scene word frequency is 10, and the historical word frequency is 8. Its final word frequency is 3×0.3+10×0.5+8×0.2=7.5, which is higher than the general word "report news" (final word frequency = 5.2), ensuring priority association in office scenarios.

[0029] S2, extracting the first several character prefixes of all words in the core corpus and grouping them according to the length of the character prefixes to obtain several prefix groups; The step S2 comprises: extracting the first 1 character prefix, the first 2 character prefix, and so on, of all words in the core corpus until the first N character prefix, and storing the character prefixes belonging to the same character length in the same prefix group to obtain N prefix groups; It should be noted that in the present application, three prefix groups are set, namely the first prefix group, the second prefix group, and the third prefix group. For example, assuming that there is a word "report analysis", the "report" prefix will be divided into the first prefix group, the "report" prefix will be divided into the second prefix group, and the "report analysis" prefix will be divided into the third prefix group.

[0030] S3, assigning an independent thread to each of the prefix groups and constructing an index mapping in the independent thread; The step S3 comprises: assigning an independent thread to each of the prefix groups, and in the independent thread, traversing all words starting with the character prefix in the core corpus for each character prefix in each of the prefix groups, and taking the traversed words as the associated words of the character prefix, and traversing and searching for any complete keyword of the associated words in the preset scene keyword library, if a corresponding complete keyword is searched in the preset scene keyword library, assigning a corresponding scene label to the associated words, if multiple corresponding complete keywords are searched in the preset scene keyword library, preferentially assigning a scene label of the source corpus to the associated words, if no corresponding complete keyword is searched in the preset scene keyword library, assigning a general scene label to the associated words, and integrating the prefix traversal, keyword extraction, and scene label assignment process in the independent thread to obtain an index mapping; Specifically, in actual situations, there is a main thread for executing the process of step S2, i.e., grouping according to the prefix length, and since there are three prefix groups, there are also three corresponding independent threads, and the three threads process the three prefix groups respectively, and perform the index mapping of "prefix→associated word→word information (final word frequency+scene label)" in the thread (the scene label is generated by keyword matching, such as "function" "report" marked as "office", and "navigation" marked as "car"), and the specific process is as follows: For each prefix (such as "report") in the current prefix group, all words (such as "report generation" and "report analysis") starting with the prefix in the core corpus are traversed to determine the "associated words" of the prefix. Meanwhile, according to the final word frequency calculated in step S1, the final word frequency of each associated word can be determined. The scene label is implemented by "preset scene keyword library + accurate character matching". Specifically, a preset scene keyword library (such as "office" scene containing 200 professional words such as "report", "function", and "formula", and "car" scene containing 150 exclusive words such as "navigation", "driving", and "destination") is constructed in advance and can be dynamically updated through a configuration file. For the associated words, "full word matching + priority judgment" is used: traverse the complete character sequence of the word. If it contains any complete keyword in the scene keyword library (such as "report generation" containing "report"), the corresponding scene label is preliminarily marked. If multiple scene keywords are matched (such as "medical report" containing "medical" and "report"), the scene attribute of the source corpus is taken as priority (such as marking "medical" if it comes from a medical corpus). If no keyword is matched, the "general" (general scene) is marked by default. It should be noted that after the execution of steps S1-S3, the index mapping of each prefix group is integrated into the same structure (such as the hierarchical structure of "prefix length → prefix → associated word → word information"), and is persisted as a three-level index file in dic.json format (to avoid repeated construction). Therefore, before the next input recommendation process, it can be determined in advance whether the three-level index file exists locally. If it exists, the file construction process is skipped. If it does not exist, the above steps are executed to complete the construction of the three-level index file. At the same time, during the subsequent intelligent input recommendation process, the memory cache area is initialized, and the prefixes with the top 1000 final word frequencies in each prefix group are loaded first. If the memory occupancy is less than 30MB, the prefixes are loaded further. Then, the input method can be started by the user, and the input state is waited for.

[0031] S4, obtaining input association query content issued by a user, and performing downgrade matching based on the input association query content according to the independent thread to obtain a matching result; The step S4 includes: S41, extracting the first N characters of the association query content as input content, and performing retrieval in the independent thread corresponding to the Nth prefix group. If the corresponding character prefix is retrieved, the output is stored in the matching result. S42, if the corresponding character prefix is not retrieved, extracting the first N-1 characters of the association query content as input content, and performing retrieval in the independent thread corresponding to the N-1th prefix group. If the corresponding character prefix is retrieved, the output is stored in the matching result. S43, if the corresponding character prefix is not retrieved, the first N-2 characters of the associated query content are extracted as input content, retrieval is performed in the independent thread corresponding to the N-2 prefix group, the degradation matching process is repeatedly executed until the first 1 character of the associated query content is extracted as input content, retrieval is performed in the independent thread corresponding to the 1 prefix group, if the corresponding character prefix is retrieved, the output is stored in the matching result; S44, if the corresponding character prefix is not retrieved, the last 1 character of the associated query content is extracted as input content, retrieval is performed in the independent thread corresponding to the 1 prefix group, if the corresponding character prefix is retrieved, the output is stored in the matching result.

[0032] Taking the user input "reporting" (office scenario) as an example, the matching process is as follows: Step 1: complete word matching (priority 1) The complete length (3 characters) of the input word "reporting" is extracted, the independent thread corresponding to the 3 prefix group is queried, if the "reporting" prefix exists, the associated vocabulary is returned and stored in the matching result; if it does not exist, step 2 is entered.

[0033] Step 2: long prefix degradation (priority 2) The first 2 characters of the input word, "report" (long prefix, ensuring semantic association), are extracted, the independent thread corresponding to the 2 prefix group is queried, if it exists, the "report generation" "report analysis" and other vocabularies are obtained and supplemented to the matching result, if the result is empty, step 3 is entered.

[0034] Step 3: short prefix completion (priority 3) If the long prefix has no result, the first 1 character of the input word, "report" (short prefix), is extracted, the independent thread corresponding to the 1 prefix group is queried, if it exists, the "report" "reporting" and other vocabularies are obtained and supplemented to the matching result, if the result is still empty, step 4 is entered.

[0035] Step 4: single character bottom (priority 4) The last 1 character of the input word ("reporting") is extracted, the independent thread corresponding to the 1 prefix group is queried, if it exists, the "analysis" "classification" and other vocabularies are obtained, and finally all the results are integrated, and the matching result can be returned (for example, "report generation" "report analysis" "analysis" is returned); It should be noted that after the user input content, it is necessary to judge whether the input memory is in the memory cache, if it is in the cache, the subsequent steps are executed, if it is not in the cache, the corresponding index data of the prefix is read from the local three-level index file and written into the memory cache, and the low-frequency prefixes in the cache are eliminated.

[0036] S5, performing a three-level index query based on the matching result and the index mapping to output a recommendation result.

[0037] The step S5 includes: extracting a corresponding character prefix from the matching result, extracting an associated vocabulary corresponding to the character prefix from the index mapping, eliminating irrelevant vocabulary in the associated vocabulary according to a scenario label to obtain a filtered vocabulary, performing a deduplication process on the filtered vocabulary and arranging the filtered vocabulary in descending order according to a corresponding final word frequency, and recommending the top several filtered vocabularies in the arrangement to obtain a recommendation result. Specifically, the scenario label "office" is used to filter the vocabulary (to exclude irrelevant vocabulary in the associated vocabulary), and then the final word frequency (the first part of the calculated weighted word frequency) of the vocabulary is arranged in descending order — for example, the final word frequency of "report generation" is 7.5, and the final word frequency of "report analysis" is 6.2. After the arrangement, "report generation" is preferred to "report analysis", and the top 5 high-frequency words are returned (to meet the requirements of "quick positioning + accurate filtering", and to ensure the matching degree of the association result and the user input intention). Meanwhile, the deduplication process here is: An empty hash table is initialized (the key is "the text content of the vocabulary", and the value is "the final word frequency of the vocabulary"), and the "key non-repetition" feature of the hash table is used to realize fast deduplication. According to the matching priority order of the above steps, each vocabulary in the result is traversed in turn, and an empty hash table is initialized with "the text content of the vocabulary" as the key and "the final word frequency of the vocabulary" as the value (the "key non-repetition" feature of the hash table is used to realize fast deduplication): if the text content of the currently traversed vocabulary does not exist in the hash table (for example, "report generation" is traversed for the first time), the text content and the final word frequency of the vocabulary are stored in the hash table; if the text content of the current vocabulary already exists in the hash table (for example, "analysis" appears repeatedly in different steps), the record stored in the hash table for the first time is directly retained — because the vocabulary stored for the first time comes from a higher-priority matching step, its semantic relevance to the user input word is stronger, and there is no need to replace and update.

[0038] Specifically, after input recommendation, it can be judged whether the user's embedded device is connected to the network. If connected to the network, the user's historical corpus is incrementally updated in the background (without affecting the current input), and if not connected to the network, the index file is run locally without function degradation.

[0039] In summary, the intelligent input recommendation method based on multi-source corpus and parallel index provided by the embodiment solves the problems of the existing deep learning model, such as “memory occupation ≥ 200MB” and “dependence on cloud computing power”, ensures stable operation of the scheme on an embedded device (vehicle terminal, smart watch) with CPU ≤ 2 cores and memory ≤ 1 GB, and discards the traditional semantic model (such as BERT and Word2Vec) to achieve precise association only through “weighted word frequency + scene label”, thereby avoiding model loading and occupying memory. The invention only retains words with a word frequency ≥ 1.0 (eliminates extremely low-frequency rare words), further compresses the index file size (from 50MB to 35MB), loads on demand when the user first queries, reduces memory occupation at startup (startup memory ≤ 30MB), caches the prefix index of the user's recent queries to memory, directly reuses subsequent queries, and avoids repeated file reading. The invention forms a complete technical chain of “corpus fusion → efficient construction → fast query → fault-tolerant matching → lightweight adaptation” through the synergistic effect of the above improvements, well solves the core pain points of the existing input method association function, and the scheme has the characteristics of “reproducible, easy to land, and wide adaptation”, and is suitable for office, vehicle, embedded and other multi-scene input requirements.

[0040] Embodiment two As Figure 2 shown, the embodiment two of the present application provides an intelligent input recommendation system based on multi-source corpus and parallel index, which comprises: A computing module 1 is configured to obtain a core corpus, calculate the weighted word frequency of the core corpus, and obtain the final word frequency. A grouping module 2 is configured to extract the first several character prefixes of all words in the core corpus, group the character prefixes according to their lengths, and obtain several prefix groups. A mapping module 3 is configured to assign an independent thread to each prefix group and construct an index mapping in the independent thread. A matching module 4 is configured to obtain the input association query content issued by the user, perform a downgrade matching based on the input association query content according to the independent thread, and obtain a matching result. A recommendation module 4 is configured to perform a three-level index query based on the matching result and the index mapping, and output a recommendation result.

[0041] The computing module 1 is configured to: Load a general basic corpus, a scene professional corpus, and a user history corpus, respectively perform text information on the general basic corpus, the scene professional corpus, and the user history corpus, remove data noise to obtain a general basic de-noised corpus, a scene professional de-noised corpus, and a user history de-noised corpus, identify a basic word frequency, a scene word frequency, and a history word frequency of each word in the general basic de-noised corpus, the scene professional de-noised corpus, and the user history de-noised corpus respectively, calculate a final word frequency of each word based on the basic word frequency, the scene word frequency, and the history word frequency , the scene word frequency , and the history word frequency . : ; , the second weight, and the third weight respectively, .

[0042] The grouping module 2 is configured to: extract a first character prefix, a second character prefix, and an Nth character prefix of all words in the core corpus in sequence, and store character prefixes of the same character length in a same prefix group to obtain N prefix groups.

[0043] The mapping module 3 is configured to: assign an independent thread to each prefix group of different character lengths, and in the independent thread, traverse all words starting with each character prefix in the core corpus and take the traversed words as associated words of the character prefix, and search for any complete keyword of the associated words in a preset scene keyword library, if a corresponding complete keyword is searched in the preset scene keyword library, assign a corresponding scene label to the associated words, if multiple corresponding complete keywords are searched in the preset scene keyword library, preferentially assign a scene label of a source corpus to the associated words, if no corresponding complete keyword is searched in the preset scene keyword library, assign a general scene label to the associated words, and integrate the prefix traversal, keyword extraction, and scene label assignment processes in the independent thread to obtain an index mapping.

[0044] The matching module 4 includes: a first matching sub-module configured to extract the first N characters of the association query content as input content, search in the independent thread corresponding to the Nth prefix group, and if a corresponding character prefix is searched, store the output in a matching result. ​The second matching sub-module is configured to extract the first N-1 characters of the input content as input content, and perform retrieval in the independent thread corresponding to the N-1th prefix group, and if the corresponding character prefix is retrieved, the output is stored in the matching result. The third matching sub-module is configured to extract the first N-2 characters of the input content as input content, and perform retrieval in the independent thread corresponding to the N-2th prefix group, and repeatedly perform the degradation matching process until the first character of the input content is extracted as input content, and retrieval is performed in the independent thread corresponding to the 1th prefix group, and if the corresponding character prefix is retrieved, the output is stored in the matching result. The fourth matching sub-module is configured to extract the last 1 character of the input content as input content, and perform retrieval in the independent thread corresponding to the 1th prefix group, and if the corresponding character prefix is retrieved, the output is stored in the matching result.

[0045] The recommendation module 5 is configured to: extract the corresponding character prefix from the matching result, extract the associated vocabulary corresponding to the corresponding character prefix from the index mapping, remove irrelevant vocabulary in the associated vocabulary according to the scene label to obtain screened vocabulary, perform deduplication processing on the screened vocabulary and arrange the screened vocabulary in descending order according to the corresponding final word frequency, recommend the first several screened vocabulary after sorting to obtain a recommendation result.

[0046] In some embodiments of the present application, the computer comprises a memory 102, a processor 101 and a computer program stored in the memory 102 and executable on the processor 101, and the processor 101 implements the intelligent input recommendation method based on multi-source corpus and parallel index as described above when executing the computer program.

[0047] Specifically, the processor 101 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0048] The memory 102 can include mass storage for data or instructions. By way of example, and not limitation, the memory 102 can include a Hard Disk Drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash drive, a Compact Disk Drive (CD Drive), a DVD Drive, a tape drive, a USB drive, or any combination of two or more of these. The memory 102 can be removable and / or non-removable (or fixed) as appropriate. The memory 102 can be internal or external as appropriate. In certain embodiments, the memory 102 is a non-volatile memory. In certain embodiments, the memory 102 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or any combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended Data Output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0049] The memory 102 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 101.

[0050] The processor 101 realizes the above-mentioned intelligent input recommendation method based on multi-source corpus and parallel index by reading and executing the computer program instructions stored in the memory 102.

[0051] In some embodiments, the computer can further include a communication interface 103 and a bus 100. As shown, the processor 101, the memory 102, and the communication interface 103 are connected through the bus 100 and complete communication with each other. Figure 3

[0052] The communication interface 103 is used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 103 can also realize data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, image / data processing workstations, etc.

[0053] ​Bus 100 includes hardware, software, or both, to couple components of computer device to each other and to couple components of computer device to other devices. Bus 100 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, a local bus, etc. By way of example and not limitation, bus 100 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 100 can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0054] The computer can acquire an intelligent input recommendation system based on multi-source corpus and parallel index, execute the intelligent input recommendation method based on multi-source corpus and parallel index, and achieve intelligent input recommendation based on multi-source corpus and parallel index.

[0055] In some embodiments of the present application, in combination with the intelligent input recommendation method based on multi-source corpus and parallel index described above, the present embodiments provide the following technical solutions: a storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the intelligent input recommendation method based on multi-source corpus and parallel index.

[0056] Those skilled in the art will appreciate that the logic and / or steps represented in the flow diagrams, or otherwise described herein, for example, can be thought of as a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination of both. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0057] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then employable by a computer.

[0058] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or combinations thereof, can be used: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), or the like.

[0059] The technical features of the above-described embodiments can be combined in any manner, and for the sake of brevity, not all possible combinations are described, however, any combination of the technical features is considered to be within the scope of the present specification.

[0060] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. An intelligent input recommendation method based on multi-source corpora and parallel indexing, characterized in that, include: Obtain the core corpus, and perform weighted word frequency calculation on the core corpus to obtain the final word frequency; Extract the first few character prefixes of all words in the core corpus and group them according to the length of the character prefixes to obtain several prefix groups; A separate thread is allocated for each of the prefix groups, and an index mapping is constructed within the separate thread; Obtain the user-submitted suggested query content, and perform downgraded matching based on the input suggested query content according to the independent thread to obtain the matching result; A three-level index query is performed based on the matching results and the index mapping to output recommendation results.

2. The intelligent input recommendation method based on multi-source corpus and parallel indexing according to claim 1, characterized in that, The step of performing weighted word frequency calculation on the core corpus to obtain the final word frequency includes: Load general basic corpus, scenario-specific corpus, and user history corpus. Perform text information processing on the general basic corpus, scenario-specific corpus, and user history corpus respectively to remove data noise, resulting in general basic denoised corpus, scenario-specific denoised corpus, and user history denoised corpus. Identify the basic word frequency, scenario word frequency, and historical word frequency of each word in the general basic denoised corpus, scenario-specific denoised corpus, and user history denoised corpus, and base the results on the basic word frequency. The frequency of the scene The historical word frequency Calculate the final word frequency for each word. : ； In the formula, They are the first weight, the second weight, and the third weight, respectively. .

3. The intelligent input recommendation method based on multi-source corpus and parallel indexing according to claim 1, characterized in that, The step of extracting the first few character prefixes of all words in the core corpus and grouping them according to the length of the character prefixes to obtain several prefix groups includes: The first character prefix, the first two character prefixes, and so on up to the first N character prefixes of all words in the core corpus are extracted sequentially. Character prefixes of the same length are stored in the same prefix group to obtain N prefix groups.

4. The intelligent input recommendation method based on multi-source corpus and parallel indexing according to claim 1, characterized in that, The step of allocating an independent thread for each prefix group and constructing an index mapping within the independent thread includes: An independent thread is allocated to each prefix group of different character lengths. Within each independent thread, for each character prefix in each prefix group, all words starting with that character prefix are traversed in the core corpus, and the traversed words are used as associated words of that character prefix. Any complete keyword of the associated words is retrieved from a preset scenario keyword library. If a single corresponding complete keyword is found in the preset scenario keyword library, a corresponding scenario tag is assigned to the associated word. If multiple corresponding complete keywords are found in the preset scenario keyword library, the scenario tag of the source corpus is assigned to the associated word first. If no corresponding complete keyword is found in the preset scenario keyword library, a general scenario tag is assigned to the associated word. The prefix traversal, keyword extraction, and scenario tag assignment processes are integrated within an independent thread to obtain an index mapping.

5. The intelligent input recommendation method based on multi-source corpus and parallel indexing according to claim 1, characterized in that, The step of performing downgraded matching based on the input association query content according to the independent thread to obtain the matching result includes: Extract the first N characters of the associated query content as input content, and perform the search in an independent thread corresponding to the Nth prefix group. If the corresponding character prefix is ​​found, the output is stored in the matching result. If no corresponding character prefix is ​​found, the first N-1 characters of the associated query content are extracted as input content, and the search is performed in an independent thread corresponding to the N-1th prefix group. If a corresponding character prefix is ​​found, the output is stored in the matching result. If no corresponding character prefix is ​​found, the first N-2 characters of the associated query content are extracted as input content, and the search is performed in an independent thread corresponding to the N-2th prefix group. The downgrade matching process is repeated until the first character of the associated query content is extracted as input content and the search is performed in an independent thread corresponding to the first prefix group. If a corresponding character prefix is ​​found, the output is stored in the matching result. If no corresponding character prefix is ​​found, the last character of the associated query content is extracted as input content, and the search is performed in an independent thread corresponding to the first prefix group. If a corresponding character prefix is ​​found, the output is stored in the matching result.

6. The intelligent input recommendation method based on multi-source corpus and parallel indexing according to claim 1, characterized in that, The step of performing a three-level index query based on the matching result and the index mapping to output the recommendation result includes: The corresponding character prefix is ​​extracted from the matching result, and the associated words corresponding to the character prefix are extracted from the index mapping. Irrelevant words in the associated words are removed according to the scene label to obtain the filtered words. The filtered words are deduplicated and sorted in descending order according to the corresponding final word frequency. The top few filtered words after sorting are recommended to obtain the recommendation result.

7. An intelligent input recommendation system based on multi-source corpora and parallel indexing, characterized in that, The system includes: The calculation module is used to acquire the core corpus and perform weighted word frequency calculation on the core corpus to obtain the final word frequency; The grouping module is used to extract the first few character prefixes of all words in the core corpus and group them according to the length of the character prefixes to obtain several prefix groups; The mapping module is used to allocate an independent thread for each prefix group and construct an index mapping within the independent thread; The matching module is used to obtain the user-submitted input association query content, and perform downgraded matching based on the input association query content according to the independent thread to obtain the matching result; The recommendation module is used to perform a three-level index query based on the matching results and the index mapping to output recommendation results.

8. The intelligent input recommendation system based on multi-source corpus and parallel indexing according to claim 7, characterized in that, The calculation module is used for: Load general basic corpus, scenario-specific corpus, and user history corpus. Perform text information processing on the general basic corpus, scenario-specific corpus, and user history corpus respectively to remove data noise, resulting in general basic denoised corpus, scenario-specific denoised corpus, and user history denoised corpus. Identify the basic word frequency, scenario word frequency, and historical word frequency of each word in the general basic denoised corpus, scenario-specific denoised corpus, and user history denoised corpus, and base the results on the basic word frequency. The frequency of the scene The historical word frequency Calculate the final word frequency for each word. : ； In the formula, They are the first weight, the second weight, and the third weight, respectively. .

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent input recommendation method based on multi-source corpus and parallel index as described in any one of claims 1 to 6.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the intelligent input recommendation method based on multi-source corpus and parallel index as described in any one of claims 1 to 6.

Images