Data index creation method, device and electronic equipment

Abstract

Embodiments of the present invention disclose a data index creation method, device, and electronic device, relating to the field of computer security technology. The method comprises: analyzing the type of original data; determining the keywords required for the secondary index of the original data based on the type of the original data; obtaining the value corresponding to each keyword required for the secondary index of the original data from the original data; combining the keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data; using the secondary index of the original data as the second keyword and the primary index of the original data as the value corresponding to the second keyword, and storing them in a target database. The present invention can store the secondary index in a database on a disk without occupying a large amount of memory resources, effectively saving costs and improving data search efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of computer security technology, and in particular relates to a data index creation method, device and electronic equipment. Background Art

[0002] In order to prevent complex business threat attacks, traditional enterprise security business systems currently need to record all log data of each computer at all times and save it for a long time. As a result, when searching and analyzing data in the future, they need to search in massive data, which takes a lot of time and cost and is not conducive to subsequent threat statistical analysis.

[0003] To improve search efficiency, secondary indexes are created. Currently, these indexes are stored in the Elasticsearch database. Elasticsearch is very resource-intensive, especially memory-intensive. However, users' computers generally have limited memory, which hinders secondary index creation and slows data searches. Summary of the Invention

[0004] In light of this, embodiments of the present invention provide a data index creation method, apparatus, and electronic device to address the problem of existing secondary index creation solutions, which consume large amounts of memory and result in slow data search. This invention can store secondary indexes in a database on disk, eliminating the need for large amounts of memory resources, effectively saving costs and improving data search efficiency.

[0005] In a first aspect, an embodiment of the present invention provides a method for creating a data index, comprising:

[0006] Types of raw data to be analyzed;

[0007] Determining keywords required for a secondary index of the original data according to the type of the original data;

[0008] Obtain the value corresponding to the keyword required for each secondary index of the original data from the original data;

[0009] Combine the keywords and their corresponding values ​​required for multiple secondary indexes of the original data to obtain the secondary index of the original data;

[0010] The secondary index of the original data is used as the second keyword and the primary index of the original data is used as the value corresponding to the second keyword, which are stored in the target database.

[0011] In conjunction with the first aspect, in a first implementation of the first aspect, combining keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data includes:

[0012] Generate keyword terminators corresponding to the keywords required for each secondary index of the original data according to the preset keyword terminator generation rules;

[0013] The keyword required for the i-th secondary index of the original data, its corresponding value, and the corresponding keyword terminator are sequentially concatenated to obtain an index string corresponding to the keyword required for the i-th secondary index of the original data; where i = 1, 2, ..., N; N is the total number of keywords required for the secondary index of the original data;

[0014] The index strings corresponding to the keywords required for all secondary indexes of the original data are connected end to end to obtain the secondary index of the original data.

[0015] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the preset keyword terminator generation rule is:

[0016] A first preset symbol is added after the keyword required for the i-th secondary index of the original data as a keyword terminator corresponding to the keyword required for the i-th secondary index of the original data.

[0017] In combination with the first implementation of the first aspect, in a third implementation of the first aspect, sequentially concatenating the keyword required for the i-th secondary index of the original data, its corresponding value, and the corresponding keyword terminator to obtain an index string corresponding to the keyword required for the i-th secondary index of the original data includes:

[0018] The keyword required for the i-th secondary index of the original data, the second preset symbol, the value corresponding to the keyword required for the i-th secondary index of the original data, the second preset symbol, and the keyword terminator corresponding to the keyword required for the i-th secondary index of the original data are connected end to end in sequence to obtain the index string corresponding to the keyword required for the i-th secondary index of the original data.

[0019] In conjunction with the first implementation of the first aspect, in a fourth implementation of the first aspect, after determining the keywords required for the secondary index of the original data according to the type of the original data, and before concatenating index character strings corresponding to all the keywords required for the secondary index of the original data, the method further includes:

[0020] Sort the keywords required for all secondary indexes of the original data in descending order of historical search frequency of the keywords required for each secondary index of the original data to obtain a keyword sorting result;

[0021] The step of connecting the index strings corresponding to all the keywords required for the secondary indexes of the original data end to end to obtain the secondary indexes of the original data includes:

[0022] The index character strings corresponding to each keyword in the keyword sorting result are sequentially connected end to end according to the keyword sorting result to obtain a secondary index of the original data.

[0023] In conjunction with the first aspect, or any one of the first to fourth implementations of the first aspect, in a fifth implementation of the first aspect, after the secondary index of the original data is used as the second keyword and the primary index of the original data is used as the value corresponding to the second keyword and stored in the target database, the method further includes:

[0024] Receive a secondary index search request; wherein the secondary index search request includes at least a target index keyword and its corresponding value;

[0025] Retrieving the target index keyword from the second keyword stored in the target database, and obtaining a value corresponding to the target secondary index including the target index keyword as the target primary index corresponding to the secondary index retrieval request;

[0026] A search is performed according to the target primary index to obtain the data requested by the secondary index search request.

[0027] In combination with the fifth implementation of the first aspect, in the sixth implementation of the first aspect, before analyzing the type of the original data, the method further includes:

[0028] Generate a primary index for the original data, and store the primary index and the original data in a primary index table corresponding to each other; wherein the primary index table uses the primary index as a first keyword and the original data as a value corresponding to the first keyword;

[0029] The keywords required for the secondary index of the original data include the name of the primary index table;

[0030] The acquiring of the value corresponding to the target secondary index including the target index keyword further includes: acquiring the name of the target primary index table recorded in the target secondary index;

[0031] The retrieval based on the target first-level index to obtain the data requested by the second-level index retrieval includes: searching the target first-level index in the first-level index table corresponding to the target first-level index table name to obtain the original data corresponding to the target first-level index as the data requested by the second-level index retrieval request.

[0032] In a second aspect, an embodiment of the present invention provides a data index creation device, comprising:

[0033] Analysis module, used to analyze the type of raw data;

[0034] A keyword determination module, configured to determine keywords required for a secondary index of the original data according to the type of the original data;

[0035] The value acquisition module is used to obtain the value corresponding to each keyword required by the secondary index of the original data from the original data;

[0036] A secondary index generation module is used to combine the keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data;

[0037] The secondary index storage module is used to use the secondary index of the original data as the second keyword and the primary index of the original data as the value corresponding to the second keyword, and store them in the target database accordingly.

[0038] In conjunction with the second aspect, in a first implementation of the second aspect, the secondary index generation module includes:

[0039] A terminator generating unit, configured to generate a keyword terminator corresponding to a keyword required for each secondary index of the original data according to a preset keyword terminator generating rule;

[0040] An index string generation unit is configured to sequentially concatenate the keyword required for the i-th secondary index of the original data, its corresponding value, and the corresponding keyword terminator to obtain an index string corresponding to the keyword required for the i-th secondary index of the original data; wherein i = 1, 2, ..., N; N is the total number of keywords required for the secondary index of the original data;

[0041] The secondary index generating unit connects the index character strings corresponding to all the keywords required for the secondary index of the original data end to end to obtain the secondary index of the original data.

[0042] In combination with the first implementation of the second aspect, in the second implementation of the second aspect, the preset keyword terminator generation rule used by the terminator generation unit is:

[0043] A first preset symbol is added after the keyword required for the i-th secondary index of the original data as a keyword terminator corresponding to the keyword required for the i-th secondary index of the original data.

[0044] In combination with the first embodiment of the second aspect, in the third embodiment of the second aspect, the index string generation unit is specifically used to connect the keyword required for the i-th secondary index of the original data, the second preset symbol, the value corresponding to the keyword required for the i-th secondary index of the original data, the second preset symbol, and the keyword terminator corresponding to the keyword required for the i-th secondary index of the original data in sequence to obtain the index string corresponding to the keyword required for the i-th secondary index of the original data.

[0045] In combination with the first implementation of the second aspect, in a fourth implementation of the second aspect, the secondary index generation module further includes:

[0046] A keyword sorting unit is used to sort the keywords required for all secondary indexes of the original data in descending order according to the historical search frequency of the keywords required for each secondary index of the original data, and obtain a keyword sorting result;

[0047] The secondary index generating unit is specifically configured to sequentially connect the index character strings corresponding to each keyword in the keyword sorting result end to end according to the keyword sorting result to obtain the secondary index of the original data.

[0048] In combination with the second aspect and any one of the first to fourth implementations of the second aspect, in a fifth implementation of the second aspect, the data index creation device further includes:

[0049] A search request receiving module, configured to receive a secondary index search request; wherein the secondary index search request at least includes a target index keyword and its corresponding value;

[0050] a primary index acquisition module, configured to retrieve the target index keyword from the second keyword stored in the target database, and obtain a value corresponding to the target secondary index including the target index keyword as the target primary index corresponding to the secondary index retrieval request;

[0051] The data acquisition module is used to search according to the target primary index to obtain the data requested by the secondary index search request.

[0052] In combination with the fifth implementation of the second aspect, in a sixth implementation of the second aspect, the data index creation device further includes:

[0053] A first-level index storage module is used to generate a first-level index of the original data and store the first-level index and the original data in a first-level index table; wherein the first-level index table uses the first-level index as a first keyword and the original data as a value corresponding to the first keyword;

[0054] The keywords required for the secondary index of the original data include the name of the primary index table;

[0055] The first-level index acquisition module is further used to obtain the target first-level index table name recorded in the target second-level index;

[0056] The data acquisition module is specifically used to retrieve the target first-level index in the first-level index table corresponding to the target first-level index table name, and obtain the original data corresponding to the target first-level index as the data requested by the second-level index retrieval request.

[0057] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; a power supply circuit for supplying power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, and is used to execute the data index creation method described in any of the aforementioned embodiments.

[0058] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the data index creation method described in any of the aforementioned embodiments.

[0059] The data index creation scheme provided by the present invention first analyzes the type of original data to obtain the keywords required for the secondary index, then obtains the value corresponding to each keyword required for the secondary index from the original data, further combines the keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data, and finally stores the secondary index as the second keyword and the primary index of the original data as the value corresponding to the second keyword in the target database. The target keyword can then be searched in the second keyword attribute stored in the target database, and when the target keyword is matched, the corresponding primary index can be directly obtained. The present invention can store the secondary index in the database on the disk without occupying a large amount of memory resources, effectively saving costs and improving data search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A flow chart of a first embodiment of a method for creating a data index provided by an embodiment of the present invention;

[0062] Figure 2A flow chart of a second embodiment of a method for creating a data index provided by an embodiment of the present invention;

[0063] Figure 3 A flow chart of a third embodiment of a method for creating a data index provided by an embodiment of the present invention;

[0064] Figure 4 A schematic structural diagram of a first embodiment of a data index creation device provided by an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of the structure of a second embodiment of a data index creation device provided by an embodiment of the present invention;

[0066] Figure 6 A schematic structural diagram of a third embodiment of a data index creation device provided by an embodiment of the present invention;

[0067] Figure 7 A schematic structural diagram of a fourth embodiment of a data index creation device provided by an embodiment of the present invention;

[0068] Figure 8 A schematic diagram of a fifth embodiment of a data index creation device provided by an embodiment of the present invention;

[0069] Figure 9 This is a structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0070] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0071] It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0072] Figure 1 This is a flow chart of a data index creation method according to an embodiment of the present invention. Figure 1 The method includes the following steps S101-S105:

[0073] S101: Analyze the type of original data.

[0074] In this embodiment, the format of the original data may be JSON, or other types such as XML, etc.; the type may be a log file of file events, registry events, etc., which has the advantage of wide applicability.

[0075] S102: Determine keywords required for the secondary index of the original data according to the type of the original data.

[0076] For example, assuming there is raw data JSON1, whose type is process type (Process), the keywords required for the secondary index of JSON1 may include: search type (search_type), the name of the stored primary index table (hbase_table), the process name (process_name), and the process path (process_path).

[0077] Preferably, keywords required for various data types may be preset, and then in S202 , keywords required for the data type may be determined according to the type of original data for which an index is currently to be generated.

[0078] S103: Obtain, from the original data, a value corresponding to a keyword required for each secondary index of the original data.

[0079] For example, in the example given in S202 , the values ​​of the keywords are parsed from JSON1, search_type=process, hbase_table=ProcessEvent, process_name=sed, and process_path= / usr / bin / sed.

[0080] S104: Combine keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data.

[0081] S105: Use the secondary index of the original data as a second keyword and the primary index of the original data as a value corresponding to the second keyword, and store them in the target database.

[0082] For example, the corresponding relationship table stored in the target database is shown in Table 1 below:

[0083] Table 1

[0084] Second keyword value Secondary index of original data 1 The primary index of the original data 1 Secondary index of original data 2 The primary index of original data 2 …… ……

[0085] A data index creation method provided by an embodiment of the present invention first analyzes the type of original data to obtain the keywords required for the secondary index, then obtains the value corresponding to each keyword required for the secondary index from the original data, further combines the keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data, and finally stores the secondary index as the second keyword and the primary index of the original data as the value corresponding to the second keyword in the target database. The target keyword can be searched in the second keyword attribute stored in the target database, and when the target keyword is matched, the corresponding primary index can be directly obtained. The present invention can store the secondary index in the database on the disk without occupying a large amount of memory resources, thus realizing large-scale and inexpensive storage, effectively saving costs and improving data search efficiency.

[0086] Figure 2 This is a flow chart of a second embodiment of a data index creation method provided by an embodiment of the present invention. Figure 2 The method includes the following steps S201-S207:

[0087] S201: Analyze the type of original data.

[0088] S202: Determine keywords required for the secondary index of the original data according to the type of the original data.

[0089] S203: Obtain, from the original data, a value corresponding to a keyword required for each secondary index of the original data.

[0090] In this embodiment, the specific implementation method of steps S201-S203 is similar to the above steps S101-S103, and will not be repeated here.

[0091] S204: Generate a keyword terminator corresponding to each keyword required for each secondary index of the original data according to a preset keyword terminator generation rule.

[0092] Preferably, the preset keyword terminator generation rule is: adding a first preset symbol after the keyword required for the i-th secondary index of the original data as the keyword terminator corresponding to the keyword required for the i-th secondary index of the original data.

[0093] For example, the first preset symbol is "|", and the generated keyword terminator is used to prevent data information from being excessively searched during subsequent searches. Assuming that the keyword required for a secondary index of the original data is "process_name", the keyword terminator corresponding to the keyword generated in S204 is "process_name|".

[0094] S205: sequentially concatenate the keyword required for the i-th secondary index of the original data, its corresponding value, and the corresponding keyword terminator to obtain an index string corresponding to the keyword required for the i-th secondary index of the original data.

[0095] Wherein, i=1, 2, ..., N; N is the total number of keywords required for the secondary index of the original data.

[0096] Preferably, this step S205 may include: connecting the keyword required for the i-th secondary index of the original data, the second preset symbol, the value corresponding to the keyword required for the i-th secondary index of the original data, the second preset symbol, and the keyword terminator corresponding to the keyword required for the i-th secondary index of the original data in sequence to obtain an index string corresponding to the keyword required for the i-th secondary index of the original data; wherein the second preset symbol is different from the first preset symbol.

[0097] In this embodiment, assuming that there is a keyword key2 required for a secondary index, its corresponding value is value2, the second preset symbol is "=", and the first preset symbol is "|", then the generated index string is key2=value2=key2|, which facilitates the subsequent acquisition of the secondary index.

[0098] S206: Connect the index character strings corresponding to the keywords required for all secondary indexes of the original data end to end to obtain the secondary index of the original data.

[0099] For example, in S206 , the index string corresponding to the keyword required for the first secondary index of the original data to the index string corresponding to the keyword required for the Nth secondary index can be connected end to end.

[0100] Preferably, before step S206, the method further includes: sorting all the keywords required for the secondary indexes of the original data in descending order according to the historical search frequency of the keywords required for each secondary index of the original data, thereby obtaining a keyword sorting result. The historical search frequency of the keywords can be obtained from historical statistics, for example, by updating the historical search frequency of the currently used search keyword each time data is retrieved. In this embodiment, step S206 includes: concatenating the index strings corresponding to each keyword in the keyword sorting result in sequence according to the keyword sorting result, thereby obtaining a secondary index of the original data.

[0101] In this embodiment, the historical search frequencies of the keywords required for each secondary index are sorted from high to low, and then the keywords with high historical search frequencies are placed in front, which will effectively improve the search efficiency.

[0102] S207: Use the secondary index of the original data as a second keyword and the primary index of the original data as a value corresponding to the second keyword, and store them in the target database.

[0103] In this embodiment, it is assumed that there is an original data JSON1, the first-level index is index1, and its type is process type (Process). The keywords required for the second-level index of JSON1 may include: search type (search_type), storage table name (hbase_table), process name (process_name), process path (process_path), and the values ​​of each keyword are parsed from JSON1. search_type = process, hbase_table = ProcessEvent, process_name = sed, process_path = / usr / bin / sed, and the generated index strings are: search_type = process = search_type |, hbase_table = ProcessEvent = hbase_table |, process_name = sed = process_name |, process_path = / usr / bin / sed = process_path |, and the finally generated second-level index is index2 = search_type = process = search_type | hbase_table = ProcessEvent = hbase_table | process_n ame=sed=process_name|process_path= / usr / bin / sed=process_path|, the value of the secondary index is the primary index index1, and finally index2 is used as the keyword and the primary index index1 is used as the corresponding value and stored in the target database, such as the HBASE database, thereby completing the creation of the secondary index, which has the advantage of simple implementation.

[0104] A data index creation method provided by an embodiment of the present invention first analyzes the type of original data to obtain the keywords and corresponding values ​​required for the secondary index, then generates keyword terminators corresponding to each keyword required for the secondary index, and then sequentially concatenates the keywords required for the secondary index, their corresponding values, and the corresponding keyword terminators to obtain a secondary index. Finally, the secondary index is used as the keyword and the primary index is used as the corresponding value to be stored in a target database. The present invention can efficiently generate secondary indexes and store them in a database on disk without occupying a large amount of memory resources, effectively saving costs and improving data search efficiency.

[0105] Figure 3 This is a flow chart of a third embodiment of a data index creation method provided by an embodiment of the present invention. Figure 3 The method includes the following steps S301-S310:

[0106] S301: Analyze the type of original data.

[0107] S302: Determine keywords required for the secondary index of the original data according to the type of the original data.

[0108] S303: Obtain, from the original data, a value corresponding to a keyword required for each secondary index of the original data.

[0109] S304: Generate a keyword terminator corresponding to each keyword required for each secondary index of the original data according to a preset keyword terminator generation rule.

[0110] S305: sequentially concatenate the keyword required for the i-th secondary index of the original data, its corresponding value, and the corresponding keyword terminator to obtain an index string corresponding to the keyword required for the i-th secondary index of the original data.

[0111] Wherein, i=1, 2, ..., N; N is the total number of keywords required for the secondary index of the original data.

[0112] S306: Connect the index character strings corresponding to the keywords required for all secondary indexes of the original data end to end to obtain the secondary index of the original data.

[0113] S307: Use the secondary index of the original data as the second keyword and the primary index of the original data as the value corresponding to the second keyword, and store them in the target database.

[0114] S308: Receive a secondary index search request.

[0115] The secondary index search request at least includes a target index keyword and its corresponding value.

[0116] S309: Retrieve the target index keyword from the second keyword stored in the target database, and obtain a value corresponding to the target secondary index including the target index keyword as the target primary index corresponding to the secondary index retrieval request.

[0117] S310: Search according to the target primary index to obtain the data requested by the secondary index search request.

[0118] As an optional embodiment, before step S301, it also includes: generating a first-level index of the original data, and storing the first-level index and the original data in a first-level index table; wherein, the first-level index table uses the first-level index as the first keyword and the original data as the value corresponding to the first keyword. In this embodiment, the keywords required for the second-level index of the original data include the name of the first-level index table. In this embodiment, step S309 also includes: obtaining the target first-level index table name recorded in the target second-level index; step S310 may include: searching the target first-level index in the first-level index table corresponding to the target first-level index table name, and obtaining the original data corresponding to the target first-level index as the data requested by the second-level index retrieval request.

[0119] For example, assuming the secondary index is index2=search_type=process=search_type|hbase_table=ProcessEvent=hbase_table|process_name=sed=process_name|process_path= / usr / bin / sed=process_path|, and the value of the secondary index is index1 (i.e., the primary index), then the search process name is data containing sed. The corresponding key2 (i.e., the secondary index) and value2 (i.e., index1) can be searched through regular matching process_name=*.sed.*=process_name. Parsing key2 can obtain the table name hbase_table=ProcessEvent, and value2 is the primary index index1. Then, JSON1 can be obtained from the ProcessEvent table through the keyword key=index1 for subsequent business. This greatly improves data search efficiency and facilitates future maintenance and expansion.

[0120] An embodiment of the present invention provides a data index creation method. After creating a secondary index and storing it in a disk database, upon receiving a secondary index retrieval request, the method searches for a target index keyword from a second keyword stored in a target database, obtains a value corresponding to a target secondary index including the target index keyword, and uses this value as the target primary index corresponding to the secondary index retrieval request. The method then searches the target primary index to obtain the data requested by the secondary index retrieval request. This method effectively improves data retrieval efficiency and facilitates future maintenance, reducing maintenance costs. As data increases, only new hard disks need to be added, achieving horizontal expansion.

[0121] Corresponding to a data index creation method provided in an embodiment of the present invention, an embodiment of the present invention also provides a data index creation device. Figure 4 This is a structural diagram of a data index creation device embodiment 1 provided by the present invention. Figure 4 As shown, the device includes:

[0122] Analysis module 1, used to analyze the type of original data;

[0123] Keyword determination module 2, used to determine keywords required for the secondary index of the original data according to the type of the original data;

[0124] The value acquisition module 3 is used to obtain the value corresponding to each keyword required by the secondary index of the original data from the original data;

[0125] Secondary index generation module 4, used to combine the keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data;

[0126] The secondary index storage module 5 is configured to use the secondary index of the original data as a second keyword and the primary index of the original data as a value corresponding to the second keyword, and store the values ​​in the target database accordingly.

[0127] The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0128] Figure 5 This is a structural diagram of a second embodiment of a data index creation device provided by an embodiment of the present invention. Figure 4 On the basis of the device structure shown, further, the secondary index generation module 4 includes:

[0129] The terminator generation unit 41 is used to generate a keyword terminator corresponding to the keyword required for each secondary index of the original data according to a preset keyword terminator generation rule; preferably, the preset keyword terminator generation rule used by the terminator generation unit 41 is: adding a first preset symbol after the keyword required for the i-th secondary index of the original data as the keyword terminator corresponding to the keyword required for the i-th secondary index of the original data.

[0130] The index string generation unit 42 is used to sequentially concatenate the keyword required for the i-th secondary index of the original data and its corresponding value and the corresponding keyword terminator to obtain the index string corresponding to the keyword required for the i-th secondary index of the original data; wherein, i=1,2,…,N; N is the total number of keywords required for the secondary index of the original data; preferably, the index string generation unit 42 is specifically used to sequentially concatenate the keyword required for the i-th secondary index of the original data, the second preset symbol, the value corresponding to the keyword required for the i-th secondary index of the original data, the second preset symbol, and the keyword terminator corresponding to the keyword required for the i-th secondary index of the original data to obtain the index string corresponding to the keyword required for the i-th secondary index of the original data; wherein, the second preset symbol is different from the first preset symbol.

[0131] The secondary index generating unit 43 connects the index character strings corresponding to all the keywords required for the secondary index of the original data end to end to obtain the secondary index of the original data.

[0132] The device of this embodiment can be used to perform Figure 2 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0133] Figure 6 This is a structural diagram of a third embodiment of a data index creation device provided by an embodiment of the present invention. Figure 5 On the basis of the device structure shown, the secondary index generation module 4 further includes:

[0134] The keyword sorting unit 44 is used to sort the keywords required for all secondary indexes of the original data in descending order of historical search frequency of the keywords required for each secondary index of the original data to obtain a keyword sorting result;

[0135] The secondary index generating unit 43 is specifically configured to sequentially connect the index character strings corresponding to each keyword in the keyword sorting result end to end according to the keyword sorting result to obtain the secondary index of the original data.

[0136] The device of this embodiment can be used to perform Figure 2 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0137] Figure 7 This is a schematic diagram of a fourth embodiment of a data index creation device provided by an embodiment of the present invention. Figure 5 On the basis of the device structure shown, the data index creation device further includes:

[0138] The search request receiving module 6 is configured to receive a secondary index search request; wherein the secondary index search request at least includes a target index keyword and its corresponding value;

[0139] A primary index acquisition module 7 is configured to retrieve the target index keyword from the second keyword stored in the target database, and obtain a value corresponding to the target secondary index including the target index keyword as the target primary index corresponding to the secondary index search request;

[0140] The data acquisition module 8 is used to perform a search based on the target primary index to obtain the data requested by the secondary index search request.

[0141] The device of this embodiment can be used to perform Figure 3 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0142] Figure 8 This is a structural diagram of a fifth embodiment of a data index creation device provided by an embodiment of the present invention. Figure 7 On the basis of the device structure shown, the data index creation device further includes:

[0143] A first-level index storage module 9 is configured to generate a first-level index of the original data and store the first-level index and the original data in a first-level index table corresponding to each other; wherein the first-level index table uses the first-level index as a first keyword and the original data as a value corresponding to the first keyword;

[0144] The keywords required for the secondary index of the original data include the name of the primary index table;

[0145] The first-level index acquisition module 7 is further used to obtain the target first-level index table name recorded in the target second-level index;

[0146] The data acquisition module 8 is specifically used to retrieve the target first-level index in the first-level index table corresponding to the target first-level index table name, and obtain the original data corresponding to the target first-level index as the data requested by the second-level index search request.

[0147] An embodiment of the present invention further provides an electronic device. Figure 9 A schematic diagram of the structure of an electronic device provided by the present invention can realize the present invention Figure 1 、 Figure 2 and Figure 3 The process of the embodiment shown is as follows: Figure 9As shown, the above-mentioned electronic device may include: a shell 91, a processor 92, a memory 93, a circuit board 94 and a power supply circuit 95, wherein the circuit board 94 is placed inside the space enclosed by the shell 91, and the processor 92 and the memory 93 are arranged on the circuit board 94; the power supply circuit 95 is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory 93 is used to store executable program code; the processor 92 runs the program corresponding to the executable program code by reading the executable program code stored in the memory 93, so as to execute a data index creation method described in any of the above-mentioned embodiments.

[0148] This electronic device exists in many forms, including but not limited to:

[0149] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0150] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0151] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video playback modules (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0152] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, System Total Wire Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0153] (5) Other electronic devices with data interaction functions.

[0154] In the fourth aspect, an embodiment of the present invention also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement any data index creation method provided by the aforementioned embodiments, thereby also achieving the corresponding technical effects, which have been described in detail above and will not be repeated here.

[0155] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0156] Each embodiment in this specification is described in a related manner. Similar portions between the embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so their description is relatively simple. For related portions, refer to the description of the method embodiments.

[0157] For the convenience of description, the above device is described as being divided into various units / modules based on their functions. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0158] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0159] The above description is merely a specific embodiment of the present invention, but the scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of the present invention. Therefore, the scope of the present invention should be based on the scope of the claims.

Claims

1. A data index creation method, characterized in that: include: Types of raw data to be analyzed; Determining keywords required for a secondary index of the original data according to the type of the original data; Obtain the value corresponding to the keyword required for each secondary index of the original data from the original data; Combine the keywords and their corresponding values ​​required for multiple secondary indexes of the original data to obtain the secondary index of the original data; The secondary index of the original data is used as the second keyword and the primary index of the original data is used as the value corresponding to the second keyword, and the values ​​are stored in the target database accordingly; The step of combining keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data includes: Generate keyword terminators corresponding to the keywords required for each secondary index of the original data according to the preset keyword terminator generation rules; The original data i The keywords required for the secondary index and their corresponding values, as well as the corresponding keyword terminators, are sequentially spliced ​​to obtain the first i The index string corresponding to the keyword required for the secondary index; i =1,2,…, N ; N The total number of keywords required for the secondary index of the original data; The index strings corresponding to the keywords required for all secondary indexes of the original data are connected end to end to obtain the secondary index of the original data.

2. The data index creation method according to claim 1, wherein: After determining the keywords required for the secondary index of the original data according to the type of the original data, and before concatenating the index character strings corresponding to all the keywords required for the secondary index of the original data, the method further includes: Sort the keywords required for all secondary indexes of the original data in descending order of historical search frequency of the keywords required for each secondary index of the original data to obtain a keyword sorting result; The step of connecting the index strings corresponding to all the keywords required for the secondary indexes of the original data end to end to obtain the secondary indexes of the original data includes: The index character strings corresponding to each keyword in the keyword sorting result are sequentially connected end to end according to the keyword sorting result to obtain a secondary index of the original data.

3. The data index creation method according to any one of claims 1 to 2, characterized in that: After the secondary index of the original data is used as the second keyword and the primary index of the original data is used as the value corresponding to the second keyword and stored in the target database, the method further includes: Receive a secondary index search request; wherein the secondary index search request includes at least a target index keyword and its corresponding value; Retrieving the target index keyword from the second keyword stored in the target database, and obtaining a value corresponding to the target secondary index including the target index keyword as the target primary index corresponding to the secondary index retrieval request; A search is performed according to the target primary index to obtain the data requested by the secondary index search request.

4. A data index creation device, characterized in that: include: Analysis module, used to analyze the type of raw data; A keyword determination module, configured to determine keywords required for a secondary index of the original data according to the type of the original data; The value acquisition module is used to obtain the value corresponding to each keyword required by the secondary index of the original data from the original data; A secondary index generation module is used to combine the keywords required for multiple secondary indexes of the original data and their corresponding values ​​to obtain the secondary index of the original data; A secondary index storage module is used to use the secondary index of the original data as a second keyword and the primary index of the original data as a value corresponding to the second keyword, and store them in the target database; The secondary index generation module includes: A terminator generating unit, configured to generate a keyword terminator corresponding to a keyword required for each secondary index of the original data according to a preset keyword terminator generating rule; Index string generation unit, used to convert the first i The keywords required for the secondary index and their corresponding values, as well as the corresponding keyword terminators, are sequentially spliced ​​to obtain the first i The index string corresponding to the keyword required for the secondary index; i =1,2,…, N ; N The total number of keywords required for the secondary index of the original data; The secondary index generating unit connects the index character strings corresponding to all the keywords required for the secondary index of the original data end to end to obtain the secondary index of the original data.

5. The data index creation device according to claim 4, characterized in that: The secondary index generation module further includes: A keyword sorting unit is used to sort the keywords required for all secondary indexes of the original data in descending order according to the historical search frequency of the keywords required for each secondary index of the original data, and obtain a keyword sorting result; The secondary index generating unit is specifically configured to sequentially connect the index character strings corresponding to each keyword in the keyword sorting result end to end according to the keyword sorting result to obtain the secondary index of the original data.

6. The data index creation device according to any one of claims 4 to 5, characterized in that: The device further comprises: A search request receiving module, configured to receive a secondary index search request; wherein the secondary index search request at least includes a target index keyword and its corresponding value; a primary index acquisition module, configured to retrieve the target index keyword from the second keyword stored in the target database, and obtain a value corresponding to the target secondary index including the target index keyword as the target primary index corresponding to the secondary index retrieval request; The data acquisition module is used to search according to the target primary index to obtain the data requested by the secondary index search request.

7. An electronic device, characterized in that: The electronic device includes: a shell, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the shell, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs the program corresponding to the executable program code by reading the executable program code stored in the memory, and is used to execute the data index creation method described in any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the data index creation method according to any one of claims 1 to 3.

Images