Data search processing method, device, electronic device and storage medium

Abstract

The present invention provides a data search processing method, device, electronic device and storage medium, which create indexes of different indexing methods for data fields and data content based on data fields representing content information and / or status information of data content. When creating an index, the update frequency of the information represented by the data field is taken into account, so that a suitable indexing method can be selected based on the update frequency, so that when information with a higher update frequency is updated, it is not necessary to pseudo-delete all information related to the data content, thereby reducing the amount of pseudo-deleted data and helping to improve search performance. Indexes of different indexing methods can also be created based on the role of the data field in data search, so that the search process does not need to obtain the required information by scanning the original document every time, thereby improving search efficiency.

Description

Technical Field

[0001] The present invention relates to the field of search engine technology, and in particular to a data search processing method, device, electronic device and storage medium. Background Art

[0002] When creating an index, existing data searches usually create an inverted index based on the terms extracted from the document and the document ID, and create a forward index based on the document ID and the content contained in the document. For example, the existing open source distributed search engine based on the lucene search kernel builds indexes based on this principle. The document contains not only data content, but also data content status information. Whether the data content or status information is updated, the original document needs to be pseudo-deleted and a new document needs to be created. As time goes by, document redundancy will increase and the index size will also increase. In extreme cases, redundancy can account for 30%-50% of the entire index, which affects search performance and increases search time.

[0003] On the other hand, during the data search process, the search field is first matched with the term in the inverted index to find the recalled document ID, and then the document corresponding to the document ID is scanned to obtain relevant information for sorting (for example, status information), thereby achieving document sorting. This process requires scanning documents to obtain relevant information for sorting, which is time-consuming and inefficient.

[0004] It can be seen that the existing method of creating indexes based on word segmentation and documents will increase pseudo-deleted data over time, affecting search performance, and the search process based on the created index in the existing technology is not conducive to improving search efficiency. Summary of the invention

[0005] The present invention provides a data search processing method, device, electronic device and storage medium, which are used to solve the problem that the existing technology creates indexes based on word segmentation and documents, which will increase pseudo-deleted data over time and affect search performance, and the process of searching based on the created index in the existing technology is not conducive to improving search efficiency. The method reduces the amount of pseudo-deleted data, improves search efficiency and optimizes search performance.

[0006] The present invention provides a data search processing method, comprising:

[0007] Obtaining indexes created for data content in different indexing modes; wherein the indexing modes are determined according to the update frequency of information represented by the data field and / or the role of the data field in data search; the data field is used to represent content information and / or status information of the data content;

[0008] Data search is performed based on the index.

[0009] According to the present invention, a data search processing method is provided. Based on the above, before obtaining the index created by different indexing methods for the data content, it also includes:

[0010] If the update frequency of the information represented by the data field is greater than the preset update frequency, a positive index is created; wherein the positive index is an index for searching the data field through the data content;

[0011] If the update frequency of the information represented by the data field is less than or equal to the preset update frequency, an inverted index is created, wherein the inverted index is an index for searching the data content through the data field.

[0012] According to the present invention, a data search processing method is provided. Based on the above, before obtaining the index created by different indexing methods for the data content, it also includes:

[0013] If the role of the data field in data search is to determine the recalled data content associated with the search field, then create an inverted index, wherein the inverted index is an index for searching the data content through the data field;

[0014] If the role of the data field in data search is not to determine the content of the recalled data, a positive index is created; wherein the positive index is an index for searching the data field through the data content.

[0015] According to the present invention, a data search processing method is provided. On the basis of the above, if the role of the data field in the data search is not to determine the content of the recalled data, a positive index is created, including:

[0016] If the function of the data field in the data search is to sort the content of the recalled data, the created forward index is stored in the cache;

[0017] If the function of the data field in data search is to read the data content, the created forward index is stored in the disk.

[0018] According to the present invention, a data search processing method is provided. Based on the above, the data search is performed according to the index, including:

[0019] Determine the recalled data content associated with the search field according to the search field and the created inverted index; wherein the inverted index is an index for searching the data content through the data field;

[0020] And / or, determining the search results in response to the search request based on the recalled data content and the created forward index; wherein the forward index is an index of the data field searched through the data content.

[0021] According to the present invention, a data search processing method is provided. Based on the above, the search results in response to the search request are determined according to the recalled data content and the created forward index, including:

[0022] Acquire a data field for sorting the recalled data content through the created forward index as a status data field;

[0023] Sorting the recalled data content according to the status data field to obtain a data content sequence;

[0024] The data content that responds to the search request is determined based on the data content sequence as the search result.

[0025] According to the present invention, a data search processing method is provided. Based on the above, determining the data content in response to the search request based on the data content sequence as the search result includes:

[0026] By determining the data content to be displayed from the data content sequence;

[0027] Each data content to be displayed is read through the created forward index to obtain the search result.

[0028] The present invention provides a data search processing device, comprising:

[0029] An acquisition module, used to acquire indexes created for data content in different indexing modes; wherein the indexing modes are determined according to the update frequency of information represented by the data field and / or the role of the data field in data search; the data field is used to represent content information and / or status information of the data content;

[0030] A search module is used to search for data according to the index.

[0031] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned data search processing methods are implemented.

[0032] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned data search processing methods are implemented.

[0033] The present invention provides a data search processing method, device, electronic device and storage medium, which create indexes for data fields and data content in different indexing methods based on data fields representing content information and / or status information of data content. When creating an index, the update frequency of the information represented by the data field is taken into account, so that a suitable indexing method can be selected based on the update frequency, so that when information with a higher update frequency is updated, it is not necessary to pseudo-delete all information related to the data content, thereby reducing the amount of pseudo-deleted data and helping to improve search performance. It is also possible to create indexes in different indexing methods based on the role of the data field in data search, so that the search process does not need to obtain the required information by scanning the original document every time, thereby improving search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is one of the flow charts of the data search and processing method provided by the present invention;

[0036] Figure 2 It is a schematic diagram of a storage method of indexes created for static data fields, dynamic data fields and supplementary data fields provided by the present invention;

[0037] Figure 3 It is a schematic diagram of the index component of the search engine provided by the present invention;

[0038] Figure 4 It is a schematic diagram of a specific process of updating an index provided by the present invention;

[0039] Figure 5 It is a schematic diagram of the search workflow provided by the present invention;

[0040] Figure 6 It is one of the structural block diagrams of the data search processing device provided by the present invention;

[0041] Figure 7 It is a schematic diagram of the physical structure of the electronic device provided by the present invention; DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Figure 1 A flow chart of the data search processing method provided in this embodiment is applied to a search engine. For example, the data search processing method can improve an existing open source distributed search engine based on the lucene search kernel so that it creates an index according to the update frequency of information represented by a data field or the role of a data field in data search. On the one hand, when the data content and its related status information are updated, there is no need to pseudo-delete the entire data content and its related status information each time, thereby reducing the update frequency and storage size of the inverted index. On the other hand, during the search process, there is no need to scan the entire data content and its related status information through the disk each time to obtain information for sorting the recalled information, thereby reducing IO overhead and improving search performance.

[0044] See also Figure 1 , the data search processing method includes:

[0045] Step 101: Obtain an index created for the data content in different indexing methods; wherein the indexing method is determined according to the update frequency of the information represented by the data field or the role of the data field in data search; the data field is used to represent the content information and / or status information of the data content.

[0046] Data content is content that is stored for subsequent search through indexes, and does not include data content status information. For example, data content is the news content itself, and does not include news content status information such as the number of clicks, views, and comments on the news content. Data fields can be fields that represent content information of the data content, such as the title of the data content, the author, and keywords extracted from the data content, and can also be fields that represent data content status information, such as the number of clicks, views, and comments on the news content.

[0047] The update frequency of the information represented by the data field refers to the frequency at which the information represented by the data field is updated. For example, the number of clicks on news content is updated 100 times a day on average. Usually, the role of data fields in data search includes determining the recalled data content associated with the search field, sorting the recalled data content, and reading the data content.

[0048] In the prior art, the data content and status information are stored as a whole in the document, and then an inverted index of word segmentation and document is established. Therefore, whether the data content or the status information is updated, the entire document needs to be pseudo-deleted. In the present application, when creating an index, the update frequency of the data field can be considered to select an appropriate indexing method, thereby reducing the frequency of pseudo-deletion of the entire document. For example, when the status information with a high update frequency is used as a data field, a forward index that searches the data field through the data content can be created, so that when the status information is updated, it is only necessary to modify the status information in the forward index without pseudo-deleting the entire document, which greatly reduces the amount of pseudo-deleted data.

[0049] You can also consider the role of data fields in data search and create indexes in different indexing methods, so that during the data search process, you can quickly obtain the required information through the index, rather than going through the entire document containing data content and data content status information each time to obtain the required information, thereby improving search efficiency. For example, in data search, the role of a data field is to sort the recalled data content. Create a positive index that searches for the data field through the data content, so that for each data content, you can quickly obtain the information used for sorting through the positive index, without scanning the entire document, thereby improving search efficiency.

[0050] Step 102: Search for data according to the index.

[0051] The data search process is usually based on a certain search field. For example, after obtaining the search statement input by the user, the search field can be extracted from the search statement, and the inverted index for searching the data content through the search field is created. The recalled data content associated with the search field is obtained by matching the search field with the data field in the inverted index, and then the information used to sort the recalled data content is quickly obtained through the forward index to achieve sorting of the recalled data content. The recalled data content is scanned as the search result based on the sorted recalled data content.

[0052] The present embodiment provides a data search processing method, which creates indexes for data fields and data content in different indexing methods based on data fields representing content information and / or status information of data content. When creating an index, the update frequency of the information represented by the data field is taken into account, so that a suitable indexing method can be selected based on the update frequency, so that when information with a higher update frequency is updated, it is not necessary to pseudo-delete all information related to the data content, thereby reducing the amount of pseudo-deleted data and helping to improve search performance. It is also possible to create indexes in different indexing methods based on the role of the data field in data search, so that the search process does not need to obtain the required information by scanning the original document every time, thereby improving search efficiency.

[0053] By setting an appropriate preset update frequency, you can determine the indexing method for creating an index while considering the update frequency of the information represented by the data field and the role of the data field in data search. For example, you can comprehensively consider the update frequency of the information represented by the data field and the role of the data field in data search, divide the data field into three types: static data field, dynamic data field, and supplementary data field, and then create an index in different indexing methods according to the type of the data field.

[0054] The update frequency of the information represented by these three types of data fields and their role in data search are as follows:

[0055] Static data fields: used to retrieve data based on search fields, with low update frequency, such as resource name, author, alias, tag, category, release time, etc.

[0056] Dynamic data fields: fields used to improve search rankings, which are updated very frequently. For example, the number of views, likes, favorites, comments, ratings, clicks, conversion rates, etc. Dynamic data types do not participate in word segmentation retrieval and are only used to improve document score weights.

[0057] Supplementary data field: A field used to supplement the search return data. It is updated frequently and does not affect search recall and sorting, but only the display of search results. Updates support real-time updates, such as cover, playback address, corner mark, update time, number of episodes, etc.

[0058] Usually, data fields that represent status information (dynamic data fields) are updated frequently, and data fields used to search for data content for display (supplementary data fields) are also updated frequently due to changes in the number of pages to be displayed, so for these two types of data fields, a forward index is created to search for the data field through the data content. However, data fields used to query data content related to the search field (static data fields) are usually data fields with a low update frequency, so an inverted index is created to search for data content through the data field.

[0059] Further, based on the above embodiments, before obtaining the index created for the data content in different indexing modes, the method further includes:

[0060] If the update frequency of the information represented by the data field is greater than the preset update frequency, a positive index is created; wherein the positive index is an index for searching the data field through the data content;

[0061] If the update frequency of the information represented by the data field is less than or equal to the preset update frequency, an inverted index is created, wherein the inverted index is an index for searching the data content through the data field.

[0062] Specifically, searching the forward index of the data field through the data content is actually searching the index of the data field through the content identifier of the data content. Searching the inverted index of the data content through the data field is actually searching the index of the content identifier of the data content through the data field.

[0063] Generally, the information represented by dynamic data fields and supplementary data fields is updated frequently, so a forward index can be created for the dynamic data fields and supplementary data fields. However, the information represented by static data fields is updated less frequently, so an inverted index can be created for the static data fields.

[0064] In this embodiment, for data fields with a high update frequency, an inverted index is not created. Therefore, when the data fields are frequently updated, it is only necessary to update the content of the data fields stored in the forward index corresponding to the data content. There is no need to update the entire data content and its status information, thereby reducing the amount of pseudo-deleted data.

[0065] Further, based on the above embodiments, before obtaining the index created for the data content in different indexing modes, the method further includes:

[0066] If the role of the data field in data search is to determine the recalled data content associated with the search field, then create an inverted index, wherein the inverted index is an index for searching the data content through the data field;

[0067] If the role of the data field in data search is not to determine the content of the recalled data, a positive index is created; wherein the positive index is an index for searching the data field through the data content.

[0068] During the search process, compared with the forward index, the inverted index can more efficiently determine the recalled data content associated with the search field. Therefore, in this embodiment, an inverted index can be created for the above-mentioned static data field, while the above-mentioned dynamic data field and supplementary data field are not used to determine the recalled data content. Therefore, a forward index can be created for the dynamic data field and the supplementary data field.

[0069] In this embodiment, indexes are created in different indexing modes according to whether the role of the data field in data search is to determine the content of the recalled data, which is beneficial to improving search efficiency and optimizing the performance of the search engine.

[0070] Further, based on the above embodiments, if the function of the data field in data search is not to determine the content of the recalled data, then creating a forward index includes:

[0071] If the function of the data field in the data search is to sort the content of the recalled data, the created forward index is stored in the cache;

[0072] If the function of the data field in data search is to read the data content, the created forward index is stored in the disk.

[0073] In order to further improve the search performance, the search can also be stored in different storage spaces according to the search process. Since the recalled data content is usually large in size, for example, 1,000 data contents are recalled, and the amount of calculation is large when all the data contents need to be sorted. At this time, if the data fields used for sorting are read from the disk, it will take a long time. In order to improve the sorting efficiency, the data fields for sorting the recalled data contents (for example, the above-mentioned dynamic data fields) are stored in the cache.

[0074] In addition, since the data content used for display is usually small, for example, it is necessary to display relevant information of the data content corresponding to the content ID of 10 data contents, the data field for reading the data content (for example, the above-mentioned supplementary data field) can be stored in the disk. Since the amount of data to be processed is small, disk storage will not affect the search performance and will not occupy additional memory resources.

[0075] In this embodiment, for indexes created for different data fields, appropriate storage locations are selected according to specific processing procedures, which is beneficial to further optimize the search process and improve search efficiency.

[0076] Specifically, Figure 2 This is a schematic diagram of the storage method of the index created for the static data field, dynamic data field and supplementary data field in this embodiment, see Figure 2 , the dictionary index (i.e., term index) used to quickly locate the search field position and the inverted index created for the static data field can be stored in the same location, so that the content ID of the data content associated with the search field can be quickly retrieved through the term index. The index created for the dynamic data field is stored in the cache in a key-value manner so that the recalled data content can be quickly sorted. The index created for the supplementary data field is saved on disk, and the content ID list of the data content is also saved on disk. Usually, after search recall, sorting, and paging, only one page of documents (usually no more than 50) is returned, so the data content to be displayed can be quickly obtained by querying the disk through the forward index.

[0077] Figure 3 A schematic diagram of the index components of the search engine provided in this embodiment, see Figure 3 , you can include the following indexes in the search engine:

[0078] Dictionary index: used to quickly determine the location of the index field Term, which can be loaded into memory.

[0079] Data field - content ID of data content (content ID is content identification) inverted index: used to find the content ID list of data content according to Term.

[0080] Content ID - positive index of data field (stored in key-value cache): used to find data content according to the content ID list of data content. Due to the use of key-value memory cache, the query speed is much higher than scanning the disk according to the positive index, which greatly improves efficiency.

[0081] Content ID - positive index of data content storage location: content ID and document location index of stored data content.

[0082] Data content on disk: The content ID and data content of the data content are stored on the disk.

[0083] Based on the indexes created for static data fields, dynamic data fields, and supplementary data fields, Figure 4 For a detailed flowchart of the index update process provided in this embodiment, see Figure 4 , the process includes the following:

[0084] Step 1: Obtain a data content update request (wherein the data content update request refers to a request to update data content or update status information of data content);

[0085] The search engine obtains the data content update request, performs parameter pre-check, and proceeds to the next step after passing.

[0086] Step 2: Determine whether the data content exists (you can determine whether the data content to be updated exists by the content ID corresponding to the data content);

[0087] Retrieve whether the data content exists based on the primary key corresponding to the data content.

[0088] If it does not exist, the new document creation process of the data content is carried out (i.e. step 3).

[0089] If the data content already exists, the document update process of the data content is followed (i.e. step 4);

[0090] Step 3: Create a new document for data content;

[0091] Classify and process all types of fields in data content:

[0092] Static data fields (used for recall, such as resource name, author, alias, tag, category, release time, etc.): segment the static data fields, build a term index, and build an inverted index of the document list corresponding to the term mapping data content.

[0093] Dynamic data fields (used to improve sorting weight, such as playback volume, likes, favorites, comments, ratings, clicks, conversion rate, etc.): Store dynamic data fields in the key-value cache of the data content.

[0094] Supplementary data fields (used to supplement information, which does not affect the recall and sorting results, such as cover, playback address, subtitle, update time, number of episodes, etc.): store the supplementary data fields in disk documents and construct a positive index of the document corresponding to the data content.

[0095] Step 4: Document update process for data content

[0096] Field classification processing based on data content request update:

[0097] Static data field: Mark the document containing the original data content as pseudo-deleted, and re-create a new document containing the updated data content, see step 3.

[0098] Dynamic data field: Updates the information of the data field in the document key-value cache.

[0099] Supplement data field: Update the information corresponding to this data field in the disk document.

[0100] Step 5: Index building is complete.

[0101] Further, based on the above embodiments, the data search according to the index includes:

[0102] Determine the recalled data content associated with the search field according to the search field and the created inverted index; wherein the inverted index is an index for searching the data content through the data field;

[0103] And / or, determining the search results in response to the search request based on the recalled data content and the created forward index; wherein the forward index is an index of the data field searched through the data content.

[0104] The search field is extracted according to the search sentence input by the user, for example, by performing word segmentation processing on the search sentence.

[0105] Specifically, based on the search field, the recalled data content is quickly determined by creating an inverted index for the static data field.

[0106] In this embodiment, during the data search process, the data content associated with the search field is first determined by matching the search field with the data field in the inverted index, and the recalled data content is quickly determined through the inverted index.

[0107] Further, based on the above embodiments, determining the search results in response to the search request according to the recalled data content and the created forward index includes:

[0108] Acquire a data field for sorting the recalled data content through the created forward index as a status data field;

[0109] Sorting the recalled data content according to the status data field to obtain a data content sequence;

[0110] The data content that responds to the search request is determined based on the data content sequence as the search result.

[0111] Specifically, by creating a positive index for the above dynamic data field, the data field used for sorting the data content can be quickly obtained, thereby achieving sorting of the recommendation degree of the data content.

[0112] In this embodiment, information for sorting the data content is quickly acquired through the forward index, thereby achieving sorting of the data content.

[0113] Further, based on the above embodiments, determining the data content in response to the search request based on the data content sequence as the search result includes:

[0114] Determining data content to be displayed from the data content sequence;

[0115] Each data content to be displayed is read through the created forward index to obtain the search result.

[0116] Specifically, the data content to be displayed is determined from the data content sequence according to the display configuration information, wherein the display configuration information includes at least one of the following: the number of data content items displayed on each page of the displayed search page, and the number of pages currently requested to be displayed.

[0117] In this embodiment, the data content to be displayed is read through the forward index to obtain the search result, thereby realizing the search process.

[0118] Figure 5 See the search workflow diagram of the search engine provided in this embodiment. Figure 5 , the search process includes:

[0119] Step 1. Receive a search request

[0120] Preprocess search request parameters and perform syntax checking.

[0121] Step 2: Perform a classified query based on the query statement type to obtain a search field for searching.

[0122] Segmentation type: segment the query words to obtain the segmented term entries.

[0123] Non-word segmentation type: For example, prefix search, retrieve the term list that meets the prefix requirements based on the dictionary index.

[0124] Step 3. Query the inverted index of the static data field

[0125] Among them, step 2 obtains a list of all terms that meet the requirements. According to the dictionary index and the inverted index of the content ID of the data field-data content, a list of all content IDs that meet the requirements is queried.

[0126] Step 4. Perform an intersection operation on the recalled content IDs

[0127] All content ID lists that meet the recall criteria are intersected and combined to obtain the final recalled content ID list, and a document matching score is given. For example, the intersection of the content IDs of the data content recalled by each search field is calculated to obtain the final recalled content ID list.

[0128] Step 5. Obtain dynamic data fields, increase document score weights, and perform comprehensive scoring and sorting.

[0129] For the content ID list recalled in step 4, query the document key-value to obtain dynamic data, such as the number of views, likes, favorites, comments, ratings, clicks, conversion rates, etc., and perform comprehensive scoring and sorting according to the matching degree and weighting rules at the time of recall. Obtain the final sorting result.

[0130] Step 6. Perform paging

[0131] The sorted results are paged according to the page number and page size of the search request to obtain a list of all content IDs searched this time (ie, a list of content IDs to be displayed).

[0132] Step 6. Query supplementary data information

[0133] For the sorted and paginated results, query the disk document based on the content ID list of all the content searched this time to obtain supplementary data information (such as cover, playback address, superscript, update time, number of episodes, etc.), supplement the information required for the complete search, and obtain the search results.

[0134] Step 7. Return to full search results

[0135] This application determines whether a field is a static data field, a dynamic data field, or a supplementary data field based on the field type and field update frequency, and establishes indexes of different indexing methods for storage; static data fields are stored in the term index and inverted index, dynamic data fields are stored in the key-value cache, and supplementary data fields are stored in the disk document list. During the search process, based on the data field-data content content ID inverted index, based on the recalled content ID list, the document key-value is queried to obtain the sorting result, and the disk document is queried based on the content ID list to obtain the complete result.

[0136] Specifically, this application defines static, dynamic, and supplementary type fields according to the different uses of the fields (recall, sorting, supplementation) and different update frequencies, and classifies and stores them. Static data is stored in the dictionary and inverted index, dynamic data is stored in the cache, and supplementary data is stored on disk (using the forward index to speed up the search). At the same time, when the document index is updated, it is processed separately according to the updated field type to reduce redundant data caused by the update. At the same time, the use of document key-value cache improves the recall speed. Only after sorting and paging, it is necessary to obtain a small amount of document content through the forward index, which greatly reduces the disk IO pressure. At the same time, the physical update method is used to reduce the redundancy of pseudo-deleted data, reduce the size of the inverted index, and the update frequency, and improve search performance.

[0137] Figure 6 For a structural block diagram of a data search processing device provided in this embodiment, see Figure 6 , including an acquisition module 601 and a search module 602, wherein,

[0138] The acquisition module 601 is used to acquire indexes created for data content in different indexing modes; wherein the indexing modes are determined according to the update frequency of the information represented by the data field and / or the role of the data field in data search; the data field is used to represent the content information and / or status information of the data content;

[0139] The search module 602 is used to search for data according to the index.

[0140] The data search processing device provided in this embodiment is applicable to the data search processing methods provided in the above embodiments, which will not be described in detail here.

[0141] The present embodiment provides a data search processing device, which creates indexes for data fields and data content in different indexing methods based on data fields representing content information and / or status information of data content. When creating an index, the update frequency of the information represented by the data field is taken into account, so that a suitable indexing method can be selected based on the update frequency, so that when information with a higher update frequency is updated, it is not necessary to pseudo-delete all information related to the data content, thereby reducing the amount of pseudo-deleted data and helping to improve search performance. Indexes can also be created in different indexing methods based on the role of data fields in data search, so that the search process does not need to obtain the required information by scanning the original document every time, thereby improving search efficiency.

[0142] According to the present invention, a data search processing device is provided. Based on the above, before obtaining the index created by different indexing methods for the data content, it also includes:

[0143] If the update frequency of the information represented by the data field is greater than the preset update frequency, a positive index is created; wherein the positive index is an index for searching the data field through the data content;

[0144] If the update frequency of the information represented by the data field is less than or equal to the preset update frequency, an inverted index is created, wherein the inverted index is an index for searching the data content through the data field.

[0145] According to the present invention, a data search processing device is provided. Based on the above, before obtaining the index created by different indexing methods for the data content, it also includes:

[0146] If the role of the data field in data search is to determine the recalled data content associated with the search field, then create an inverted index, wherein the inverted index is an index for searching the data content through the data field;

[0147] If the role of the data field in data search is not to determine the content of the recalled data, a positive index is created; wherein the positive index is an index for searching the data field through the data content.

[0148] According to the present invention, a data search processing device is provided. Based on the above, if the role of the data field in the data search is not to determine the content of the recalled data, a positive index is created, including:

[0149] If the function of the data field in the data search is to sort the content of the recalled data, the created forward index is stored in the cache;

[0150] If the function of the data field in data search is to read the data content, the created forward index is stored in the disk.

[0151] According to the present invention, a data search processing device is provided. Based on the above, the data search is performed according to the index, including:

[0152] Determine the recalled data content associated with the search field according to the search field and the created inverted index; wherein the inverted index is an index for searching the data content through the data field;

[0153] And / or, determining the search results in response to the search request based on the recalled data content and the created forward index; wherein the forward index is an index of the data field searched through the data content.

[0154] According to the present invention, a data search processing device is provided. Based on the above, determining the search results in response to the search request according to the recalled data content and the created positive index includes:

[0155] Acquire a data field for sorting the recalled data content through the created forward index as a status data field;

[0156] Sorting the recalled data content according to the status data field to obtain a data content sequence;

[0157] The data content that responds to the search request is determined based on the data content sequence as the search result.

[0158] According to the present invention, there is provided a data search processing device. Based on the above, the step of determining the data content in response to the search request based on the data content sequence as the search result includes:

[0159] By determining the data content to be displayed from the data content sequence;

[0160] Each data content to be displayed is read through the created forward index to obtain the search result.

[0161] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the data search processing method, which includes:

[0162] Obtaining indexes created for data content in different indexing modes; wherein the indexing modes are determined according to the update frequency of information represented by the data field and / or the role of the data field in data search; the data field is used to represent content information and / or status information of the data content;

[0163] Data search is performed based on the index.

[0164] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0165] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the computer can perform a data search processing method, the method comprising:

[0166] Obtaining indexes created for data content in different indexing modes; wherein the indexing modes are determined according to the update frequency of information represented by the data field and / or the role of the data field in data search; the data field is used to represent content information and / or status information of the data content;

[0167] Data search is performed based on the index.

[0168] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, a data search processing method is implemented, the method comprising:

[0169] Obtaining indexes created for data content in different indexing modes; wherein the indexing modes are determined according to the update frequency of information represented by the data field and / or the role of the data field in data search; the data field is used to represent content information and / or status information of the data content;

[0170] Data search is performed based on the index.

[0171] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data search processing method, characterized in that: include: Obtain indexes created for data content in different indexing modes; wherein the indexing mode is determined according to the update frequency of the information represented by the data field and the role of the data field in data search; the data field is used to represent the content information and / or status information of the data content; wherein the update frequency of the information represented by the data field is the frequency at which the information represented by the data field is updated; the role of the data field in data search includes determining the recalled data content associated with the search field, sorting the recalled data content and reading the data content; the data field is divided into static data field, dynamic data field and supplementary data field; the static data field is used to recall the data content according to the search field; the dynamic data field is used to improve the search ranking field; the supplementary data field is used to supplement the search return data field; based on the dynamic data field, a forward index is created to search the data field through the data content; based on the static data field, a reverse index is created to search the data content through the data field; Searching for data according to the index; Before obtaining the index created by different indexing methods for the data content, it also includes: If the update frequency of the information represented by the data field is greater than the preset update frequency, a positive index is created; wherein the positive index is an index for searching the data field through the data content; If the update frequency of the information represented by the data field is less than or equal to the preset update frequency, an inverted index is created, wherein the inverted index is an index for searching the data content through the data field.

2. The data search processing method according to claim 1, characterized in that: Before obtaining the index created by different indexing methods for the data content, it also includes: If the role of the data field in data search is to determine the recalled data content associated with the search field, then create an inverted index, wherein the inverted index is an index for searching the data content through the data field; If the role of the data field in data search is not to determine the content of the recalled data, a positive index is created; wherein the positive index is an index for searching the data field through the data content.

3. The data search processing method according to claim 2, characterized in that: If the function of the data field in the data search is not to determine the content of the recalled data, then creating a forward index includes: If the function of the data field in the data search is to sort the content of the recalled data, the created forward index is stored in the cache; If the function of the data field in data search is to read the data content, the created forward index is stored in the disk.

4. The data search processing method according to claim 1, characterized in that: The performing data search according to the index comprises: Determine the recalled data content associated with the search field according to the search field and the created inverted index; wherein the inverted index is an index for searching the data content through the data field; And / or, determining search results in response to the search request based on the recalled data content and the created forward index; wherein the forward index is an index for searching the data field through the data content.

5. The data search processing method according to claim 4, characterized in that: The step of determining search results in response to the search request according to the recalled data content and the created forward index includes: Acquire a data field for sorting the recalled data content through the created forward index as a status data field; Sorting the recalled data content according to the status data field to obtain a data content sequence; The data content that responds to the search request is determined based on the data content sequence as the search result.

6. The data search processing method according to claim 5, characterized in that: The step of determining the data content in response to the search request based on the data content sequence as the search result includes: By determining the data content to be displayed from the data content sequence; Each data content to be displayed is read through the created forward index to obtain the search result.

7. A data search processing device, characterized in that: include: An acquisition module, used to acquire indexes created for data content in different indexing modes; wherein the indexing mode is determined according to the update frequency of the information represented by the data field and / or the role of the data field in data search; the data field is used to represent the content information and / or status information of the data content; wherein the update frequency of the information represented by the data field is the frequency at which the information represented by the data field is updated; the role of the data field in data search includes determining the recalled data content associated with the search field, sorting the recalled data content and reading the data content; the data field is divided into a static data field, a dynamic data field and a supplementary data field; the static data field is used to recall the data content according to the search field; the dynamic data field is used to improve the search ranking field; the supplementary data field is used to supplement the search return data field; based on the dynamic data field, a forward index is created to search the data field through the data content; based on the static data field, a reverse index is created to search the data content through the data field; A search module, used for searching data according to the index; Before obtaining the index created by different indexing methods for the data content, it also includes: If the update frequency of the information represented by the data field is greater than the preset update frequency, a positive index is created; wherein the positive index is an index for searching the data field through the data content; If the update frequency of the information represented by the data field is less than or equal to the preset update frequency, an inverted index is created, wherein the inverted index is an index for searching the data content through the data field.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the data search processing method according to any one of claims 1 to 6 are implemented.

9. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data search processing method according to any one of claims 1 to 6 are implemented.

Images