Data search method and device

Abstract

The present invention discloses a data search method and device, wherein the method comprises: obtaining to-be-searched word data; inputting the to-be-searched word data into a pre-established Whoosh library, and outputting first related search content corresponding to the to-be-searched word data; analyzing the to-be-searched word data using a word vector tool, Gensim, to obtain a word vector corresponding to the to-be-searched word data; inputting the word vector corresponding to the to-be-searched word data into a pre-established Faiss library, and outputting an index ID corresponding to the word vector; inputting the index ID into a pre-established Whoosh library, and outputting second related search content corresponding to the index ID; generating an associative word corresponding to the to-be-searched word data based on the first related search content and the second related search content; and performing a data search based on the associative word. The present invention can avoid the problems of inaccurate search results and very low search efficiency, and improve user experience.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a data search method and device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] With the rapid development of enterprises and applications, business needs and data storage models are becoming increasingly complex. There is more and more collaboration between data and the amount of data is increasing. Search is facing problems such as strong dependencies between services, data silos, and performance bottlenecks caused by excessive data volume.

[0004] Traditional search relies on basic queries against relational databases to retrieve the information users seek. These queries can only search data from a single application. Existing relational databases, such as MySQL, are often used for this purpose. As business grows, model complexity increases, and data volumes grow, relational table models become increasingly complex, gradually straining database performance. This increase in data volume and model complexity means that database indexes are no longer sufficient to address query performance, resulting in inaccurate search results, very low search efficiency, and a poor user experience.

[0005] To address the above issues, no effective solutions have been proposed so far. Summary of the Invention

[0006] The present invention provides a data search method to solve the problem that existing search methods that rely on database indexes are prone to inaccurate search results and very low search efficiency, thereby improving user experience. The method includes:

[0007] Get the data of the search term;

[0008] Input the to-be-searched word data into a pre-established whoosh library, and output first related search content corresponding to the to-be-searched word data;

[0009] Use the word vector tool gensim to analyze the word data to be searched and obtain the word vector corresponding to the word data to be searched;

[0010] Input the word vector corresponding to the word data to be searched into the pre-established faiss library, and output the index id corresponding to the word vector;

[0011] Input the index ID into a pre-established whoosh library, and output the second related search content corresponding to the index ID;

[0012] generating an associative word corresponding to the to-be-searched word data according to the first related search content and the second related search content;

[0013] Perform data search based on the associated words.

[0014] Furthermore, a Whoosh library is pre-established in the following manner: obtaining application processing data in the ES library; determining corresponding index fields based on the application processing data; and establishing a Whoosh library based on the application processing data and the corresponding index fields.

[0015] Furthermore, the to-be-searched word data is input into a pre-established whoosh library, and the first related search content corresponding to the to-be-searched word data is output, including:

[0016] Matching the search term data with the application processing data in the whoosh library;

[0017] According to the matching result, the first related search content corresponding to the to-be-searched word data is output.

[0018] Furthermore, the word vector tool gensim is used to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched, including:

[0019] Input the word data to be searched into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the word data to be searched;

[0020] Input the word data to be searched into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the word data to be searched;

[0021] The word vector corresponding to the word data to be searched is obtained based on the latent semantic index vector and word2vec word vector corresponding to the word data to be searched.

[0022] Furthermore, the faiss library is pre-established in the following manner: obtaining application processing data in the ES library; using the word vector tool gensim to analyze the application processing data to obtain the word vector corresponding to the application processing data; and establishing the faiss library based on the word vector corresponding to the application processing data.

[0023] Furthermore, the word vector tool gensim is used to analyze the application processing data to obtain the word vector corresponding to the application processing data, including:

[0024] Input the application processing data into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the application processing data;

[0025] Input the application processing data into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the application processing data;

[0026] According to the latent semantic index vector and word2vec word vector corresponding to the application processing data, the word vector corresponding to the application processing data is obtained.

[0027] Furthermore, the word vector corresponding to the word data to be searched is input into the pre-established faiss library, and the index id corresponding to the word vector is output, including:

[0028] Query the faiss database for one or more word vectors that are closest to the word vector corresponding to the word data to be searched;

[0029] Determine the index id corresponding to the one or more word vectors.

[0030] Furthermore, generating an associative word corresponding to the to-be-searched word data according to the first related search content and the second related search content includes:

[0031] Performing deduplication and fusion processing on the first related search content and the second related search content;

[0032] According to the result of the deduplication and fusion processing, an associative word corresponding to the word data to be searched is generated.

[0033] In an embodiment of the present invention, a data search device is provided to solve the problem that existing search methods relying on database indexes are prone to inaccurate search results and very low search efficiency, thereby improving user experience. The device includes:

[0034] A data acquisition module is used to obtain data of the word to be searched;

[0035] A first output module is configured to input the to-be-searched word data into a pre-established whoosh library and output first related search content corresponding to the to-be-searched word data;

[0036] An analysis module is used to analyze the word data to be searched using the word vector tool gensim to obtain the word vector corresponding to the word data to be searched;

[0037] The second output module is used to input the word vector corresponding to the search word data into the pre-established faiss library and output the index ID corresponding to the word vector;

[0038] A third output module is used to input the index ID into a pre-established whoosh library and output the second related search content corresponding to the index ID;

[0039] An associative word generation module, configured to generate an associative word corresponding to the to-be-searched word data according to the first related search content and the second related search content;

[0040] The data search module is used to perform data search based on the associative words.

[0041] Furthermore, a Whoosh library is pre-established in the following manner: obtaining application processing data in the ES library; determining corresponding index fields based on the application processing data; and establishing a Whoosh library based on the application processing data and the corresponding index fields.

[0042] Furthermore, the first output module is further configured to:

[0043] Matching the search term data with the application processing data in the whoosh library;

[0044] According to the matching result, the first related search content corresponding to the to-be-searched word data is output.

[0045] Furthermore, the analysis module is further configured to:

[0046] Input the word data to be searched into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the word data to be searched;

[0047] Input the word data to be searched into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the word data to be searched;

[0048] The word vector corresponding to the word data to be searched is obtained based on the latent semantic index vector and word2vec word vector corresponding to the word data to be searched.

[0049] Furthermore, the faiss library is pre-established in the following manner: obtaining application processing data in the ES library; using the word vector tool gensim to analyze the application processing data to obtain the word vector corresponding to the application processing data; and establishing the faiss library based on the word vector corresponding to the application processing data.

[0050] Furthermore, the word vector tool gensim is used to analyze the application processing data to obtain the word vector corresponding to the application processing data, including:

[0051] Input the application processing data into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the application processing data;

[0052] Input the application processing data into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the application processing data;

[0053] According to the latent semantic index vector and word2vec word vector corresponding to the application processing data, the word vector corresponding to the application processing data is obtained.

[0054] Furthermore, the second output module is further configured to:

[0055] Query the faiss database for one or more word vectors that are closest to the word vector corresponding to the word data to be searched;

[0056] Determine the index id corresponding to the one or more word vectors.

[0057] Furthermore, the associative word generation module is further used to:

[0058] Performing deduplication and fusion processing on the first related search content and the second related search content;

[0059] According to the result of the deduplication and fusion processing, an associative word corresponding to the word data to be searched is generated.

[0060] A computer device is also provided in an embodiment of the present invention. The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned data search method is implemented.

[0061] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program for executing the above-mentioned data search method.

[0062] The embodiment of the present invention obtains data of words to be searched; inputs the data of words to be searched into a pre-established whoosh library, and outputs first related search content corresponding to the data of words to be searched; uses the word vector tool gensim to analyze the data of words to be searched, and obtains word vectors corresponding to the data of words to be searched; inputs the word vectors corresponding to the data of words to be searched into a pre-established faiss library, and outputs the index id corresponding to the word vector; inputs the index id into the pre-established whoosh library, and outputs second related search content corresponding to the index id; generates associative words corresponding to the data of words to be searched based on the first related search content and the second related search content; and performs data search based on the associative words. Compared with the existing solution of relying on database index for data search, the embodiment of the present invention can effectively solve the problems of inaccurate search results and very low search efficiency, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0064] Figure 1 A flow chart of a data search method provided in an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of a data search device provided in an embodiment of the present invention;

[0066] Figure 3 The figure is a schematic diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0068] As mentioned earlier, traditional search retrieves the information users want by performing basic queries on relational databases, and can only search data within a single application. As businesses grow, model complexity increases, and data volumes grow, relational table models become increasingly complex, and database performance becomes increasingly strained. The increase in data volume and model complexity means that database indexes are no longer the only solution for query performance, resulting in inaccurate search results and very low search efficiency. With the rapid development of enterprises and applications, business needs, and increasingly complex data storage models, data collaboration is increasing, and data volumes are increasing. Search faces problems such as strong inter-service dependencies, data silos, and performance bottlenecks caused by excessive data volumes. Existing relational databases, such as MySQL, face difficulties resolving performance bottlenecks through indexing and other methods when faced with big data and complex data models, leading to long query times and a poor user experience.

[0069] In order to solve the problem that existing search methods rely on database indexes and are prone to inaccurate search results and very low search efficiency, and to improve user experience, an embodiment of the present invention provides a data search method. Figure 1 A data search method flow chart is provided in an embodiment of the present invention. Figure 1 As shown, the following steps are included:

[0070] Step 101: Obtain search term data;

[0071] Step 102: input the search term data into a pre-established Whoosh library, and output first related search content corresponding to the search term data;

[0072] Step 103: Analyze the word data to be searched using the word vector tool gensim to obtain the word vector corresponding to the word data to be searched;

[0073] Step 104: input the word vector corresponding to the to-be-searched word data into the pre-established faiss library, and output the index ID corresponding to the word vector;

[0074] Step 105: input the index ID into a pre-established Whoosh library, and output the second related search content corresponding to the index ID;

[0075] Step 106: Generate an associative word corresponding to the to-be-searched word data based on the first related search content and the second related search content;

[0076] Step 107: Perform data search based on the associated words.

[0077] Depend on Figure 1 As shown, the embodiment of the present invention obtains the data of the word to be searched; inputs the data of the word to be searched into a pre-established whoosh library, and outputs the first related search content corresponding to the data of the word to be searched; uses the word vector tool gensim to analyze the data of the word to be searched, and obtains the word vector corresponding to the data of the word to be searched; inputs the word vector corresponding to the data of the word to be searched into a pre-established faiss library, and outputs the index id corresponding to the word vector; inputs the index id into the pre-established whoosh library, and outputs the second related search content corresponding to the index id; generates the associative words corresponding to the data of the word to be searched according to the first related search content and the second related search content; and performs data search according to the associative words. Compared with the existing solution of relying on database index for data search, the embodiment of the present invention can effectively solve the problems of inaccurate search results and very low search efficiency, and improve the user experience.

[0078] In an embodiment, data of a word to be searched is obtained. In an embodiment, the data of the word to be searched is input into a pre-established whoosh library, and first related search content corresponding to the data of the word to be searched is output.

[0079] In this embodiment, a Whoosh library is pre-established in the following manner: obtaining application processing data in the ES library; determining corresponding index fields based on the application processing data; and establishing a Whoosh library based on the application processing data and the corresponding index fields.

[0080] In this embodiment, the word data to be searched is input into a pre-established whoosh library, and the first related search content corresponding to the word data to be searched is output, including: matching the word data to be searched with the application processing data in the whoosh library; and outputting the first related search content corresponding to the word data to be searched based on the matching result.

[0081] In the embodiment, the word vector tool gensim is used to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched.

[0082] In this embodiment, the word vector tool gensim is used to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched, including: inputting the word data to be searched into the lsi model in the word vector tool gensim, and outputting the latent semantic index vector corresponding to the word data to be searched; inputting the word data to be searched into the word2vec model in the word vector tool gensim, and outputting the word2vec word vector corresponding to the word data to be searched; according to the latent semantic index vector and the word2vec word vector corresponding to the word data to be searched, the word vector corresponding to the word data to be searched is obtained.

[0083] In the embodiment, the word vector corresponding to the word data to be searched is input into a pre-established faiss library, and the index id corresponding to the word vector is output;

[0084] In this embodiment, the faiss library is pre-established in the following manner: obtaining application processing data in the ES library; using the word vector tool gensim to analyze the application processing data to obtain the word vector corresponding to the application processing data; and establishing the faiss library based on the word vector corresponding to the application processing data.

[0085] In this embodiment, the word vector tool gensim is used to analyze the application processing data to obtain the word vector corresponding to the application processing data, including: inputting the application processing data into the lsi model in the word vector tool gensim, and outputting the latent semantic index vector corresponding to the application processing data; inputting the application processing data into the word2vec model in the word vector tool gensim, and outputting the word2vec word vector corresponding to the application processing data; and obtaining the word vector corresponding to the application processing data according to the latent semantic index vector and the word2vec word vector corresponding to the application processing data.

[0086] In this embodiment, the word vector corresponding to the word data to be searched is input into a pre-established faiss library, and the index id corresponding to the word vector is output, including: querying the faiss library for one or more word vectors that are closest to the word vector corresponding to the word data to be searched; and determining the index id corresponding to the one or more word vectors.

[0087] In an embodiment, the index ID is input into a pre-established Whoosh library, and a second related search content corresponding to the index ID is output. Based on the first related search content and the second related search content, an associative word corresponding to the search term data is generated. Data search is performed based on the associative word.

[0088] In this embodiment, based on the first related search content and the second related search content, associative words corresponding to the word data to be searched are generated, including: deduplication and fusion processing of the first related search content and the second related search content; and generation of associative words corresponding to the word data to be searched based on the result of the deduplication and fusion processing.

[0089] A specific embodiment is given below to illustrate the specific application of the data search method in the embodiment of the present invention. In this specific embodiment, the application-processed data is collected in its own repository, and then the data collected by the application is processed, analyzed, and processed to realize the search function. The application-processed data is processed by the application according to the format provided by the unified search, and the data model is processed and placed in a specific table. The unified search loads the data through the tool and stores it in its own ES library. The data storage processing synchronizes part of the content in the ES library to whoosh through the synchronization tool and adds an index field; for each data, the word vector tool gensim is used to calculate the lsi latent semantic index vector and the word2vec word vector, and the vectors are merged and stored in the faiss library. Among them, the lsi model and the word2vec model are batch training models. After parsing, processing, and storing the data processed by the application, the unified search performs the data search process. The process of searching for associative and related terms: The search term is first matched with relevant search content in Whoosh. The word vector tool gensim is used to obtain and calculate the LSI latent semantic index vector and word2vec word vector. The index ID corresponding to the n nearest neighbor word vectors is queried in the Faiss library. The search content corresponding to this index is queried in Whoosh to generate related searches. The related searches are deduplicated and merged with the Whoosh direct match results to generate associative terms, which are then stored in ES. The process of comprehensive and precise queries: The input search term is segmented. The word segmenter will segment the search term based on the user input, and then match the segmented content with synonyms and other word analysis methods. The result set is then searched in ES, and the hit results are returned.

[0090] Based on the field of big data technology, the embodiments of the present invention implement query methods such as real-time matching of search content (with associative word completion), comprehensive multi-product queries, and precise single-product queries. These methods enable efficient search even with large data volumes and support optimization methods such as word segmentation and synonyms for user-input words. This system provides an efficient and accurate search system for local government affairs. By segmenting user-input words and configuring dictionaries, the system enables word association and matching for user-input words. This significantly improves search efficiency and user experience in scenarios with large data volumes. The embodiments of the present invention replace existing search methods, enabling fast and effective search in scenarios with large data volumes and complex data models. This improves search hit rates and search scope, thereby enhancing the user experience. Through a unified search platform, complex data structure queries and multi-application data queries can be implemented, improving user experience and reducing inter-service dependencies. The unified search system utilizes a next-generation system as its foundation, based on the Spring framework, the ElasticSearch search, analysis, and storage tool, and the Logstash collection, conversion, and storage tool. Logstash collects, converts, and stores data in ElasticSearch, completing the collection of unified search information. Its most prominent feature is its ability to significantly improve query efficiency and search results. The system also supports simultaneous querying of data from multiple services. It also enhances query performance through methods like word segmentation, thereby improving the user experience.

[0091] Based on the same inventive concept, the present invention also provides a data search device in the following embodiments. Since the principle of the device to solve the problem is similar to that of the data search method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0092] Figure 2 A schematic diagram of a data search device is provided in an embodiment of the present invention. Figure 2 As shown, the data search device provided in the embodiment of the present invention may include:

[0093] The data acquisition module 201 is used to obtain the data of the word to be searched;

[0094] A first output module 202 is configured to input the to-be-searched word data into a pre-established Whoosh library and output first related search content corresponding to the to-be-searched word data;

[0095] An analysis module 203 is configured to analyze the to-be-searched word data using a word vector tool, gensim, to obtain a word vector corresponding to the to-be-searched word data;

[0096] The second output module 204 is used to input the word vector corresponding to the to-be-searched word data into the pre-established faiss library and output the index ID corresponding to the word vector;

[0097] A third output module 205 is configured to input the index ID into a pre-established whoosh library and output the second related search content corresponding to the index ID;

[0098] An associative word generating module 206 is configured to generate an associative word corresponding to the to-be-searched word data according to the first related search content and the second related search content;

[0099] The data search module 207 is used to perform data search based on the associative words.

[0100] In one embodiment, the whoosh library is pre-built as follows:

[0101] Obtain application processing data in the ES library;

[0102] Determining corresponding index fields according to the application processing data;

[0103] A whoosh library is established based on the application processing data and corresponding index fields.

[0104] In one embodiment, the first output module 202 is further configured to:

[0105] Matching the search term data with the application processing data in the whoosh library;

[0106] According to the matching result, the first related search content corresponding to the to-be-searched word data is output.

[0107] In one embodiment, the analysis module 203 is further configured to:

[0108] Input the word data to be searched into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the word data to be searched;

[0109] Input the word data to be searched into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the word data to be searched;

[0110] The word vector corresponding to the word data to be searched is obtained based on the latent semantic index vector and word2vec word vector corresponding to the word data to be searched.

[0111] In one embodiment, the faiss library is pre-built as follows:

[0112] Obtain application processing data in the ES library;

[0113] Analyze the application processing data using the word vector tool gensim to obtain the word vector corresponding to the application processing data;

[0114] A faiss library is established based on the word vectors corresponding to the application processing data.

[0115] In one embodiment, the word vector tool gensim is used to analyze the application processing data to obtain the word vector corresponding to the application processing data, including:

[0116] Input the application processing data into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the application processing data;

[0117] Input the application processing data into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the application processing data;

[0118] According to the latent semantic index vector and word2vec word vector corresponding to the application processing data, the word vector corresponding to the application processing data is obtained.

[0119] In one embodiment, the second output module 204 is further configured to:

[0120] Query the faiss database for one or more word vectors that are closest to the word vector corresponding to the word data to be searched;

[0121] Determine the index id corresponding to the one or more word vectors.

[0122] In one embodiment, the associative word generation module 206 is further configured to:

[0123] Performing deduplication and fusion processing on the first related search content and the second related search content;

[0124] According to the result of the deduplication and fusion processing, an associative word corresponding to the word data to be searched is generated.

[0125] In summary, the embodiment of the present invention obtains the data of the word to be searched; inputs the data of the word to be searched into a pre-established whoosh library, and outputs the first related search content corresponding to the data of the word to be searched; uses the word vector tool gensim to analyze the data of the word to be searched, and obtains the word vector corresponding to the data of the word to be searched; inputs the word vector corresponding to the data of the word to be searched into a pre-established faiss library, and outputs the index id corresponding to the word vector; inputs the index id into the pre-established whoosh library, and outputs the second related search content corresponding to the index id; generates the associative words corresponding to the data of the word to be searched according to the first related search content and the second related search content; and performs data search according to the associative words. Compared with the existing solution of relying on database index for data search, the embodiment of the present invention can effectively solve the problems of inaccurate search results and very low search efficiency, and improve the user experience.

[0126] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device embodiment for implementing all or part of the above data search method. The electronic device specifically includes the following contents:

[0127] A processor, a memory, a communications interface, and a bus; wherein the processor, the memory, and the communications interface communicate with each other via the bus; the communications interface is used to implement information transmission between related devices; the electronic device can be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited thereto. In this embodiment, the electronic device can be implemented with reference to the embodiment for implementing the above-mentioned data search method and the embodiment for implementing the above-mentioned data search device, the contents of which are incorporated herein and repeated parts are not repeated.

[0128] Figure 3 FIG. 1 is a schematic diagram of a system structure of an electronic device provided in an embodiment of the present invention. Figure 3 As shown, the electronic device 30 may include a processor 301 and a memory 302; the memory 302 is coupled to the processor 301. Figure 3 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0129] In one embodiment, the functions implemented by the data search method can be integrated into the processor 301. The processor 301 can be configured to perform the following control: obtain the word data to be searched; input the word data to be searched into a pre-established whoosh library, and output the first related search content corresponding to the word data to be searched; use the word vector tool gensim to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched; input the word vector corresponding to the word data to be searched into a pre-established faiss library, and output the index ID corresponding to the word vector; input the index ID into the pre-established whoosh library, and output the second related search content corresponding to the index ID; generate an associative word corresponding to the word data to be searched based on the first related search content and the second related search content; and perform data search based on the associative word.

[0130] From the above, it can be seen that the electronic device provided in the embodiment of the present invention obtains the word data to be searched; inputs the word data to be searched into a pre-established whoosh library, and outputs the first related search content corresponding to the word data to be searched; uses the word vector tool gensim to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched; inputs the word vector corresponding to the word data to be searched into a pre-established faiss library, and outputs the index id corresponding to the word vector; inputs the index id into a pre-established whoosh library, and outputs the second related search content corresponding to the index id; generates associative words corresponding to the word data to be searched based on the first related search content and the second related search content; and performs data search based on the associative words.

[0131] The electronic device provided in the embodiment of the present invention can solve the problems of inaccurate search results and very low search efficiency in existing search methods that rely on database indexes, thereby improving user experience.

[0132] In another embodiment, the data search device may be configured separately from the processor 301. For example, the data search device may be configured as a chip connected to the processor 301, and the functions of the data search method may be implemented under the control of the processor.

[0133] like Figure 3 As shown, the electronic device 30 may further include: a communication module 303, an input unit 304, an audio processing unit 305, a display 306, and a power supply 303. It is worth noting that the electronic device 30 does not necessarily have to include Figure 3 In addition, the electronic device 30 may also include Figure 3 For components not shown, reference may be made to the prior art.

[0134] like Figure 3 As shown, the processor 301 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The processor 301 receives inputs and controls the operations of various components of the electronic device 30 .

[0135] Memory 302 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information and may also store programs that execute the relevant information. Processor 301 may execute the programs stored in memory 302 to implement information storage or processing.

[0136] Input unit 304 provides input to processor 301. Input unit 304 may be, for example, a keypad or touch input device. Power supply 303 is used to provide power to electronic device 30. Display 306 is used to display objects such as images and text. This display may be, for example, an LCD display, but is not limited thereto.

[0137] The memory 302 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), or a SIM card. Alternatively, it may be a memory that retains information even when power is off, can be selectively erased, and is provided with more data. Examples of such memory are sometimes referred to as EPROMs. The memory 302 may also be some other type of device. The memory 302 includes a buffer memory 3021 (sometimes referred to as a buffer). The memory 302 may include an application / function storage unit 3022 for storing application programs and function programs or processes for executing the operation of the electronic device 30 via the processor 301.

[0138] The memory 302 may also include a data storage unit 3023 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 3024 of the memory 302 may include various driver programs for the electronic device for communication functions and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0139] The communication module 303 is a transmitter / receiver that sends and receives signals via the antenna 308. The communication module (transmitter / receiver) 303 is coupled to the processor 301 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0140] Based on different communication technologies, multiple communication modules 303 can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module. The communication module (transmitter / receiver) 303 is also coupled to a speaker 309 and a microphone 310 via an audio processing unit 305 to provide audio output via the speaker 309 and receive audio input from the microphone 310, thereby implementing common telecommunication functions. The audio processing unit 305 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processing unit 305 is coupled to the processor 301, enabling local recording via the microphone 310 and playback of stored audio via the speaker 309.

[0141] An embodiment of the present invention also provides a computer-readable storage medium for implementing all the steps of the data search method in the above embodiment. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, all the steps of the data search method in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented: obtaining the word data to be searched; inputting the word data to be searched into a pre-established whoosh library, and outputting the first related search content corresponding to the word data to be searched; using the word vector tool gensim to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched; inputting the word vector corresponding to the word data to be searched into a pre-established faiss library, and outputting the index id corresponding to the word vector; inputting the index id into a pre-established whoosh library, and outputting the second related search content corresponding to the index id; generating associative words corresponding to the word data to be searched based on the first related search content and the second related search content; and performing data search based on the associative words.

[0142] As can be seen from the above, the computer-readable storage medium provided in the embodiments of the present invention obtains word data to be searched; inputs the word data to be searched into a pre-established whoosh library, and outputs a first related search content corresponding to the word data to be searched; uses the word vector tool gensim to analyze the word data to be searched to obtain a word vector corresponding to the word data to be searched; inputs the word vector corresponding to the word data to be searched into a pre-established faiss library, and outputs an index id corresponding to the word vector; inputs the index id into a pre-established whoosh library, and outputs a second related search content corresponding to the index id; generates an associative word corresponding to the word data to be searched based on the first related search content and the second related search content; and performs data search based on the associative word.

[0143] The computer-readable storage medium provided in the embodiments of the present invention can solve the problems of inaccurate search results and very low search efficiency in existing search methods that rely on database indexes, thereby improving user experience.

[0144] Although the present invention provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative work. The order of steps listed in the embodiments is only one way of executing the steps among many steps and does not represent the only execution order. When an actual device or client product is executed, the method can be executed sequentially or in parallel according to the embodiments or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).

[0145] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, devices (systems), or computer program products. Thus, the embodiments of this specification may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0149] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between the various embodiments may be referred to in conjunction with each other. Each embodiment focuses on the differences from the other embodiments. In particular, the system embodiments, since they are generally similar to the method embodiments, are described more simply. For relevant details, refer to the description of the method embodiments. In this document, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Terms such as "upper" and "lower" to indicate orientations or positional relationships are based on those shown in the accompanying drawings and are intended solely to facilitate the description of the present invention and simplify the description. They are not intended to indicate or imply that the devices or elements referred to must have, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Unless otherwise expressly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be a communication between the two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention is not limited to any single aspect, nor to any single embodiment, nor to any combination and / or permutation of these aspects and / or embodiments. Each aspect and / or embodiment of the present invention may be used alone or in combination with one or more other aspects and / or other embodiments.

[0150] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A data search method, characterized in that: include: Get the data of the search term; Input the search term data into a pre-established Whoosh library, and output first related search content corresponding to the search term data; wherein the Whoosh library is pre-established in the following manner: obtain application-processed data from an ES library; determine corresponding index fields based on the application-processed data; and establish a Whoosh library based on the application-processed data and the corresponding index fields; Use the word vector tool gensim to analyze the word data to be searched and obtain the word vector corresponding to the word data to be searched; Input the word vector corresponding to the word data to be searched into the pre-established faiss library, and output the index ID corresponding to the word vector; wherein the faiss library is pre-established in the following manner: obtain application processing data from the ES library; use the word vector tool gensim to analyze the application processing data to obtain the word vector corresponding to the application processing data; establish the faiss library based on the word vector corresponding to the application processing data; Input the index ID into a pre-established whoosh library, and output the second related search content corresponding to the index ID; generating an associative word corresponding to the to-be-searched word data according to the first related search content and the second related search content; Performing data search based on the associated words; The method includes inputting the to-be-searched word data into a pre-established Whoosh library and outputting first related search content corresponding to the to-be-searched word data, comprising: matching the to-be-searched word data with application-processed data in the Whoosh library; and outputting the first related search content corresponding to the to-be-searched word data based on the matching result; The word vector tool gensim is used to analyze the word data to be searched to obtain the word vector corresponding to the word data to be searched, including: Input the word data to be searched into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the word data to be searched; Input the word data to be searched into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the word data to be searched; Obtaining a word vector corresponding to the to-be-searched word data according to the latent semantic index vector and word2vec word vector corresponding to the to-be-searched word data; The method includes inputting the word vector corresponding to the word data to be searched into a pre-established faiss library and outputting the index ID corresponding to the word vector, including: searching the faiss library for one or more word vectors that are closest to the word vector corresponding to the word data to be searched; and determining the index ID corresponding to the one or more word vectors. Among them, based on the first related search content and the second related search content, generating associative words corresponding to the word data to be searched, including: deduplication and fusion processing of the first related search content and the second related search content; generating associative words corresponding to the word data to be searched based on the results of the deduplication and fusion processing.

2. The data search method according to claim 1, wherein: The word vector tool gensim is used to analyze the application processing data to obtain the word vector corresponding to the application processing data, including: Input the application processing data into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the application processing data; Input the application processing data into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the application processing data; According to the latent semantic index vector and word2vec word vector corresponding to the application processing data, the word vector corresponding to the application processing data is obtained.

3. A data search device, characterized in that: include: A data acquisition module is used to obtain data of the word to be searched; A first output module is configured to input the to-be-searched word data into a pre-established whoosh library and output first related search content corresponding to the to-be-searched word data; An analysis module is used to analyze the word data to be searched using the word vector tool gensim to obtain the word vector corresponding to the word data to be searched; The second output module is used to input the word vector corresponding to the search word data into the pre-established faiss library and output the index ID corresponding to the word vector; A third output module is used to input the index ID into a pre-established whoosh library and output the second related search content corresponding to the index ID; An associative word generation module, configured to generate an associative word corresponding to the to-be-searched word data according to the first related search content and the second related search content; A data search module, used for performing data search based on the associative words; The Whoosh library is pre-established in the following manner: obtaining application-processed data in the ES library; determining corresponding index fields based on the application-processed data; and establishing the Whoosh library based on the application-processed data and the corresponding index fields. The first output module is further configured to: match the to-be-searched word data with the application-processed data in the Whoosh library; and output first related search content corresponding to the to-be-searched word data based on the matching result; The analysis module is further used to: input the word data to be searched into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the word data to be searched; input the word data to be searched into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the word data to be searched; obtain the word vector corresponding to the word data to be searched based on the latent semantic index vector and the word2vec word vector corresponding to the word data to be searched; The faiss library is pre-established as follows: obtain application processing data from the ES library; use the word vector tool gensim to analyze the application processing data to obtain word vectors corresponding to the application processing data; establish the faiss library based on the word vectors corresponding to the application processing data; The second output module is further used to: query the faiss library for one or more word vectors that are closest to the word vector corresponding to the word data to be searched; determine the index ID corresponding to the one or more word vectors; The associative word generation module is further used to: perform deduplication and fusion processing on the first related search content and the second related search content; and generate associative words corresponding to the to-be-searched word data according to the result of the deduplication and fusion processing.

4. The data search device according to claim 3, wherein: The word vector tool gensim is used to analyze the application processing data to obtain the word vector corresponding to the application processing data, including: Input the application processing data into the LSI model in the word vector tool gensim, and output the latent semantic index vector corresponding to the application processing data; Input the application processing data into the word2vec model in the word vector tool gensim, and output the word2vec word vector corresponding to the application processing data; According to the latent semantic index vector and word2vec word vector corresponding to the application processing data, the word vector corresponding to the application processing data is obtained.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data search method according to any one of claims 1 to 2 is implemented.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for executing the data search method according to any one of claims 1 to 2.

Images