Search system establishment method, device, electronic device and readable storage medium

Abstract

The present invention relates to data processing, and discloses a method for establishing a search system, including: receiving an initial table building statement fed back by a first client user based on a table building statement template; parsing the initial table building statement to obtain the table name of the table to be built and the field type of the table header field, and converting the initial table building statement into a target table building statement based on the mapping relationship between the field type of the table building statement template and the standard field type of the search engine; establishing an index library based on the target table building statement; respectively obtaining program templates corresponding to the index building component, the search component, and the deletion component, generating the index building component, the search component, and the deletion component based on the target table building statement and the program template, and importing the source data into the index library based on the index building component to obtain a target search system. The present invention also provides a search system establishment device, an electronic device, and a readable storage medium. The present invention reduces the difficulty of search system development and improves development efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a search system establishment method, device, electronic device and readable storage medium. Background Art

[0002] A complete search system includes data source, index service, and search service. Different business data search systems correspond to different data sources, different index processes, different search logics, and log monitoring services. When a new business data search system needs to be developed, a complete set of codes needs to be developed, which makes the development cycle long, the development difficulty high, and the development efficiency low. Therefore, a search system establishment method is urgently needed to reduce the difficulty of search system development and improve development efficiency. Summary of the invention

[0003] In view of the above content, it is necessary to provide a method for establishing a search system, aiming to reduce the difficulty of developing the search system and improve the development efficiency.

[0004] The search system establishment method provided by the present invention comprises:

[0005] In response to a search system establishment request issued by a user based on a first client, a table establishment statement template is sent to the first client, and an initial table establishment statement fed back by the first client based on the table establishment statement template is received;

[0006] Parsing the initial table creation statement to obtain the table name and field types of the table header fields of the table to be created, and converting the initial table creation statement into a target table creation statement based on a mapping relationship between the field types of the table creation statement template and the standard field types of the search engine;

[0007] Based on the target table creation statement and the table name, the search system is established to request allocation of an idle search engine server cluster and to establish an index library;

[0008] Obtain program templates corresponding to indexing components, search components and deletion components respectively, generate indexing components, search components and deletion components based on the target table creation statement and the program templates, import source data corresponding to the search system establishment request into the index library based on the indexing components, and obtain the target search system.

[0009] Optionally, the generating of the indexing component, the searching component and the deleting component based on the target table creation statement and the program template includes:

[0010] Parse the target table creation statement to obtain the table header field name and format of the table to be created, and generate an entity class based on the table header field name and format;

[0011] Generate an indexing component based on the program template corresponding to the entity class and the indexing component;

[0012] Extracting a list of field names whose attributes are searchable fields from the target table creation statement, and generating a search component based on the list of field names, the entity class, and a program template corresponding to the search component;

[0013] A deletion component is generated based on the entity class and the program template corresponding to the deletion component.

[0014] Optionally, the method further includes:

[0015] When receiving an index library expansion request issued by the user based on the second client, displaying an expansion configuration interface on the display interface of the second client, and receiving a target index library and a field to be sharded input by the user based on the expansion configuration interface;

[0016] The key value of the field to be sharded is input into a preset calculation formula for calculation, and the data in the target index library is distributed to different nodes of the search engine server cluster based on the calculated value.

[0017] Optionally, the preset calculation formula is:

[0018] K%N ==0 (1)

[0019] K%N ==1 (2)

[0020] …

[0021] K%N==(N-1)(N)

[0022] Among them, K is the key value of the field to be sharded, and N is the total number of nodes after expansion;

[0023] Data satisfying formula (1) will be stored in the first node, data satisfying formula (2) will be stored in the second node, and so on, data satisfying formula (N) will be stored in the Nth node.

[0024] Optionally, the method further includes:

[0025] The resource utilization rate of the server to which the search system belongs is obtained in real time or periodically, and when the resource utilization rate exceeds a first threshold, current limiting and degradation processing is performed on the search system.

[0026] Optionally, the method further includes:

[0027] When a data write request from a user based on a third client is received, it is determined whether the amount of data to be written is greater than a second threshold. If it is determined to be greater than the second threshold, an asynchronous write operation is performed on the data to be written.

[0028] Optionally, the program template corresponding to the index building component includes the API interface name written into the index library, the source data name, the field version information and the action occurrence time information;

[0029] The program template corresponding to the search component includes sorting parameters, paging parameters, search fields and return fields.

[0030] In order to solve the above problem, the present invention further provides a search system establishment device, the device comprising:

[0031] A response module, used to respond to a search system establishment request issued by a user based on a first client, send a table establishment statement template to the first client, and receive an initial table establishment statement fed back by the first client based on the table establishment statement template;

[0032] A parsing module, used to parse the initial table creation statement, obtain the table name of the table to be created and the field type of the table header field, and convert the initial table creation statement into a target table creation statement based on the mapping relationship between the field type of the table creation statement template and the standard field type of the search engine;

[0033] An allocation module, used to allocate an idle search engine server cluster to the search system based on the target table creation statement and the table name and to establish an index library;

[0034] A generation module is used to respectively obtain program templates corresponding to indexing components, search components and deletion components, generate indexing components, search components and deletion components based on the target table creation statement and the program templates, and import the source data corresponding to the search system establishment request into the index library based on the indexing component to obtain the target search system.

[0035] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0036] at least one processor; and,

[0037] a memory communicatively connected to the at least one processor; wherein,

[0038] The memory stores a search system establishment program that can be executed by the at least one processor, and the search system establishment program is executed by the at least one processor so that the at least one processor can execute the above-mentioned search system establishment method.

[0039] In order to solve the above problems, the present invention also provides a computer-readable storage medium, on which a search system establishment program is stored. The search system establishment program can be executed by one or more processors to implement the above search system establishment method.

[0040] Compared with the prior art, the present invention first sends a table building statement template to the first client, receives an initial table building statement fed back by the first client user based on the table building statement template, and converts the initial table building statement into a target table building statement based on the mapping relationship between the field type of the table building statement template and the standard field type of the search engine; then, allocates an idle search engine server cluster, and establishes an index library based on the target table building statement. The present invention provides a table building statement template, so that the user does not need to learn the index library building method additionally, and can establish the index library by only filling in the corresponding parameters in the table building statement template, which greatly improves the efficiency of index library establishment; finally, based on the target table building statement and the program templates corresponding to the index building component, search component and deletion component, the index building component, search component and deletion component are generated, and the source data is imported into the index library based on the index building component to obtain the target search system. Therefore, the present invention reduces the difficulty of search system development and improves development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of a method for establishing a search system provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of modules of a search system establishment device provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the structure of an electronic device for implementing a method for establishing a search system provided by an embodiment of the present invention;

[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] It should be noted that the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0047] The present invention provides a method for establishing a search system. Figure 1 FIG. 1 is a flow chart of a method for establishing a search system provided by an embodiment of the present invention. The method can be executed by an electronic device, and the electronic device can be implemented by software and / or hardware.

[0048] In this embodiment, the search system establishment method includes:

[0049] S1. In response to a search system establishment request issued by a user based on a first client, a table establishment statement template is sent to the first client, and an initial table establishment statement fed back by the first client based on the table establishment statement template is received.

[0050] At present, ES (ElasticSearch, search engine framework) database building statements are usually used to build index libraries (index libraries are similar to data tables). However, ES database building statements are relatively complex and have a large number of statements. This embodiment provides a table building statement template, the syntax of which is MySQL syntax, which is a more common syntax, so that developers do not need to learn ES database building methods separately, and MySQL syntax can use a more concise language to implement table building functions (through verification, for a certain table building requirement, MySQL table building statements only have ten lines, while ES table building statements require more than seventy lines). The table building statement template using MySQL syntax can greatly improve table building efficiency.

[0051] S2. Parse the initial table creation statement to obtain the table name and field types of the table header fields of the table to be created, and convert the initial table creation statement into a target table creation statement based on a mapping relationship between the field types of the table creation statement template and the standard field types of the search engine.

[0052] By parsing the initial table creation statement, the table name, field name, field type and field format of the table to be created can be obtained. These contents are filled in the table creation statement template by the first client user. The field type includes character type, numeric type, etc., and the field format is the format of the value corresponding to the field. At the same time, the initial table creation statement also includes whether each field needs word segmentation, whether it needs to be stored, and whether it is a searchable field.

[0053] The standard field type of the search engine is the field type of the ES table creation statement, and the generated target table creation statement is the ES table creation statement. Through the mapping relationship between the field type of the table creation statement template and the standard field type of the search engine, the initial table creation statement in MySQL syntax can be converted into a target table creation statement in ES format.

[0054] S3: Based on the target table creation statement and the table name, the search system is requested to allocate an idle search engine server cluster and establish an index library.

[0055] The generated index library is similar to a data table, and the field names are the field names in the header part of the data table, that is, the generated index library is a blank data table containing only the header.

[0056] In this embodiment, an idle search engine server cluster node may be randomly allocated to the search system establishment request to establish an index library, or the index library may be established on a cluster node specified by the user.

[0057] S4. Obtain program templates corresponding to indexing components, search components and deletion components respectively, generate indexing components, search components and deletion components based on the target table creation statement and the program templates, import the source data corresponding to the search system establishment request into the index library based on the indexing components, and obtain the target search system.

[0058] The generating of indexing components, searching components and deleting components based on the target table creation statement and the program template includes:

[0059] A11, parsing the target table creation statement to obtain the table header field name and format of the table to be created, and generating an entity class based on the table header field name and format;

[0060] The format of the header field is a specific configuration of the field. For example, a field of the date type has a formate attribute, which indicates the format in which the value of the field is input and output.

[0061] The entity class defines the method for obtaining the values ​​of each field in the table header of the index library. Through the entity class, the corresponding values ​​of each field in the index library can be captured from the source data. The generation of the entity class (model class) is to map the index library in ES to a POJO class in Java (which can be directly generated according to the target table creation statement).

[0062] A12, generating an indexing component based on the program template corresponding to the entity class and the indexing component;

[0063] The indexing component includes an indexing API component and an indexing response component. The indexing API component is used to import data and store the data in an index library, and the indexing response component is used to output indexing response result information (response success or failure, response time, and response failure reason).

[0064] The program template corresponding to the index building component defines the API interface name for writing to the index library (automatically generated by the system based on the index library name and operation type (add, delete, modify, and query)), the source data name, field version information (the field version will be increased by one after each update to solve the problem of inconsistent data update results caused by concurrent access), and timestamp action occurrence time information.

[0065] A13, extracting a list of field names whose attributes are searchable fields from the target table creation statement, and generating a search component based on the list of field names, the entity class, and a program template corresponding to the search component;

[0066] For example, there is an index attribute in the target table creation statement, which indicates whether the field needs to be indexed (whether it is searchable). When the index attribute is true, it means it is searchable; when the index attribute is false, it means it is not searchable.

[0067] The search component includes a search API component and a search response component. The search API component is used to perform search operations, and the search response component is used to store retrieved data. If an execution error occurs, an error code and error message will be returned.

[0068] The program template corresponding to the search component defines sorting parameters, paging parameters, search fields and return fields.

[0069] A14. Generate a deletion component based on the entity class and the program template corresponding to the deletion component.

[0070] The deletion component includes a deletion API component and a deletion response component. The deletion API component is used to perform the deletion operation, and the deletion response component is used to record the deletion result (deletion success or failure, if the deletion fails, an error code and error message will be returned).

[0071] The indexing component, search component and deletion component are functional components of the search system. According to the indexing component, the source data can be imported into the constructed index library. At this time, the search system is completed. Subsequent users can update the data in the index library through the indexing component, search the data corresponding to the search field in the index library through the search component, and delete the data to be deleted through the deletion component.

[0072] In this embodiment, the method further includes:

[0073] B11. When receiving an index library expansion request issued by the user based on the second client, displaying an expansion configuration interface on the display interface of the second client, and receiving a target index library and a field to be sharded input by the user based on the expansion configuration interface;

[0074] B12. Input the key value of the field to be sharded into a preset calculation formula for calculation, and distribute the data in the target index library to different nodes of the search engine server cluster based on the calculated value.

[0075] In this embodiment, the preset calculation formula is:

[0076] K%N ==0 (1)

[0077] K%N ==1 (2)

[0078] …

[0079] K%N==(N-1)(N)

[0080] Among them, K is the key value of the field to be sharded, and N is the total number of nodes after expansion;

[0081] Data satisfying formula (1) may be stored in the first node, data satisfying formula (2) may be stored in the second node, ..., data satisfying formula (N) may be stored in the Nth node.

[0082] For example, if the "hospital" index library was originally stored on node A of the ES cluster, and now the "hospital" index library needs to be expanded to node B (that is, stored on both nodes A and B), then N is 2. The data corresponding to no remainder when K is divided by 2 can be stored on node A, and the data corresponding to a remainder of 1 when K is divided by 2 can be stored on node B.

[0083] The new data of the index library is directly stored on nodes A and B according to the above rules. The original old data will also be split again according to the rules and stored on the two nodes respectively.

[0084] In another embodiment of the present invention, the method further comprises:

[0085] The resource utilization rate of the server to which the search system belongs is obtained in real time or periodically, and when the resource utilization rate exceeds a first threshold, current limiting and degradation processing is performed on the search system.

[0086] The flow limiting may be to limit the maximum concurrent access volume; the downgrading may be to downgrade complex operations to ordinary operations (for example, the product page originally displays the inventory in real time, but when the server operating load is too high, it can be downgraded to a periodically updated display of the inventory).

[0087] In this embodiment, the method further includes:

[0088] When a data write request from a user based on a third client is received, it is determined whether the amount of data to be written is greater than a second threshold. If it is determined to be greater than the second threshold, an asynchronous write operation is performed on the data to be written.

[0089] It can be seen from the above embodiments that the search system establishment method proposed by the present invention first sends a table building statement template to the first client, receives the initial table building statement fed back by the first client user based on the table building statement template, and converts the initial table building statement into a target table building statement based on the mapping relationship between the field type of the table building statement template and the standard field type of the search engine; then, allocates an idle search engine server cluster, and establishes an index library based on the target table building statement. The present invention provides a table building statement template, so that the user does not need to learn the index library building method additionally, and only needs to fill in the corresponding parameters in the table building statement template to achieve the establishment of the index library, which greatly improves the efficiency of index library establishment; finally, based on the target table building statement and the program templates corresponding to the index building component, search component and deletion component, generate the index building component, search component and deletion component, and import the source data into the index library based on the index building component to obtain the target search system. Therefore, the present invention reduces the difficulty of search system development and improves development efficiency.

[0090] like Figure 2 FIG. 1 is a schematic diagram of modules of a search system establishing device provided by an embodiment of the present invention.

[0091] The search system establishment device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the search system establishment device 100 can include a response module 110, a parsing module 120, an allocation module 130 and a generation module 140. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0092] In this embodiment, the functions of each module / unit are as follows:

[0093] The response module 110 is used to respond to the search system establishment request issued by the user based on the first client, send a table creation statement template to the first client, and receive an initial table creation statement fed back by the first client based on the table creation statement template.

[0094] At present, ES (ElasticSearch, search engine framework) database building statements are usually used to build index libraries (index libraries are similar to data tables). However, ES database building statements are relatively complex and have a large number of statements. This embodiment provides a table building statement template, the syntax of which is MySQL syntax, which is a more common syntax, so that developers do not need to learn ES database building methods separately, and MySQL syntax can use a more concise language to implement table building functions (through verification, for a certain table building requirement, MySQL table building statements only have ten lines, while ES table building statements require more than seventy lines). The table building statement template using MySQL syntax can greatly improve table building efficiency.

[0095] The parsing module 120 is used to parse the initial table creation statement to obtain the table name and field types of the table header fields to be created, and convert the initial table creation statement into a target table creation statement based on the mapping relationship between the field types of the table creation statement template and the standard field types of the search engine.

[0096] By parsing the initial table creation statement, the table name, field name, field type and field format of the table to be created can be obtained. These contents are filled in the table creation statement template by the first client user. The field type includes character type, numeric type, etc., and the field format is the format of the value corresponding to the field. At the same time, the initial table creation statement also includes whether each field needs word segmentation, whether it needs to be stored, and whether it is a searchable field.

[0097] The standard field type of the search engine is the field type of the ES table creation statement, and the generated target table creation statement is the ES table creation statement. Through the mapping relationship between the field type of the table creation statement template and the standard field type of the search engine, the initial table creation statement in MySQL syntax can be converted into a target table creation statement in ES format.

[0098] The allocation module 130 is used to allocate an idle search engine server cluster to the search system establishment request based on the target table creation statement and the table name and to establish an index library.

[0099] The generated index library is similar to a data table, and the field names are the field names in the header part of the data table, that is, the generated index library is a blank data table containing only the header.

[0100] In this embodiment, an idle search engine server cluster node may be randomly allocated to the search system establishment request to establish an index library, or the index library may be established on a cluster node specified by the user.

[0101] The generation module 140 is used to respectively obtain the program templates corresponding to the indexing component, the search component and the deletion component, generate the indexing component, the search component and the deletion component based on the target table creation statement and the program template, and import the source data corresponding to the search system establishment request into the index library based on the indexing component to obtain the target search system.

[0102] The generating of indexing components, searching components and deleting components based on the target table creation statement and the program template includes:

[0103] A21, parsing the target table creation statement to obtain the table header field name and format of the table to be created, and generating an entity class based on the table header field name and format;

[0104] The format of the header field is a specific configuration of the field. For example, a field of the date type has a formate attribute, which indicates the format in which the value of the field is input and output.

[0105] The entity class defines the method for obtaining the values ​​of each field in the table header of the index library. Through the entity class, the corresponding values ​​of each field in the index library can be captured from the source data. The generation of the entity class (model class) is to map the index library in ES to a POJO class in Java (which can be directly generated according to the target table creation statement).

[0106] A22, generating an indexing component based on the program template corresponding to the entity class and the indexing component;

[0107] The indexing component includes an indexing API component and an indexing response component. The indexing API component is used to import data and store the data in an index library, and the indexing response component is used to output indexing response result information (response success or failure, response time, and response failure reason).

[0108] The program template corresponding to the index building component defines the API interface name for writing to the index library (automatically generated by the system based on the index library name and operation type (add, delete, modify, and query)), the source data name, field version information (the field version will be increased by one after each update to solve the problem of inconsistent data update results caused by concurrent access), and timestamp action occurrence time information.

[0109] A23, extracting a list of field names whose attributes are searchable fields from the target table creation statement, and generating a search component based on the list of field names, the entity class, and a program template corresponding to the search component;

[0110] For example, there is an index attribute in the target table creation statement, which indicates whether the field needs to be indexed (whether it is searchable). When the index attribute is true, it means it is searchable; when the index attribute is false, it means it is not searchable.

[0111] The search component includes a search API component and a search response component. The search API component is used to perform search operations, and the search response component is used to store retrieved data. If an execution error occurs, an error code and error message will be returned.

[0112] The program template corresponding to the search component defines sorting parameters, paging parameters, search fields and return fields.

[0113] A24. Generate a deletion component based on the program template corresponding to the entity class and the deletion component.

[0114] The deletion component includes a deletion API component and a deletion response component. The deletion API component is used to perform the deletion operation, and the deletion response component is used to record the deletion result (deletion success or failure, if the deletion fails, an error code and error message will be returned).

[0115] The indexing component, search component and deletion component are functional components of the search system. According to the indexing component, the source data can be imported into the constructed index library. At this time, the search system is completed. Subsequent users can update the data in the index library through the indexing component, search the data corresponding to the search field in the index library through the search component, and delete the data to be deleted through the deletion component.

[0116] In this embodiment, the generating module 140 is further used for:

[0117] B21. When receiving an index library expansion request issued by the user based on the second client, displaying an expansion configuration interface on the display interface of the second client, and receiving a target index library and a field to be sharded input by the user based on the expansion configuration interface;

[0118] B22. Input the key value of the field to be sharded into a preset calculation formula for calculation, and distribute the data in the target index library to different nodes of the search engine server cluster based on the calculated value.

[0119] In this embodiment, the preset calculation formula is:

[0120] K%N ==0 (1)

[0121] K%N ==1 (2)

[0122] …

[0123] K%N==(N-1)(N)

[0124] Among them, K is the key value of the field to be sharded, and N is the total number of nodes after expansion;

[0125] Data satisfying formula (1) may be stored in the first node, data satisfying formula (2) may be stored in the second node, ..., data satisfying formula (N) may be stored in the Nth node.

[0126] For example, if the "hospital" index library was originally stored on node A of the ES cluster, and now the "hospital" index library needs to be expanded to node B (that is, stored on both nodes A and B), then N is 2. The data corresponding to no remainder when K is divided by 2 can be stored on node A, and the data corresponding to a remainder of 1 when K is divided by 2 can be stored on node B.

[0127] The new data of the index library is directly stored on nodes A and B according to the above rules. The original old data will also be split again according to the rules and stored on the two nodes respectively.

[0128] In another embodiment of the present invention, the generating module 140 is further configured to:

[0129] The resource utilization rate of the server to which the search system belongs is obtained in real time or periodically, and when the resource utilization rate exceeds a first threshold, current limiting and degradation processing is performed on the search system.

[0130] The flow limiting may be to limit the maximum concurrent access volume; the downgrading may be to downgrade complex operations to ordinary operations (for example, the product page originally displays the inventory in real time, but when the server operating load is too high, it can be downgraded to a periodically updated display of the inventory).

[0131] In this embodiment, the generating module 140 is further used for:

[0132] When a data write request from a user based on a third client is received, it is determined whether the amount of data to be written is greater than a second threshold. If it is determined to be greater than the second threshold, an asynchronous write operation is performed on the data to be written.

[0133] As can be seen from the above embodiments, the search system establishment device 100 proposed by the present invention first sends a table building statement template to the first client, receives the initial table building statement fed back by the first client user based on the table building statement template, and converts the initial table building statement into a target table building statement based on the mapping relationship between the field type of the table building statement template and the standard field type of the search engine; then, allocates an idle search engine server cluster, and establishes an index library based on the target table building statement. The present invention provides a table building statement template, so that the user does not need to learn the index library building method additionally, and only needs to fill in the corresponding parameters in the table building statement template to achieve the establishment of the index library, which greatly improves the efficiency of index library establishment; finally, based on the target table building statement and the program templates corresponding to the index building component, search component and deletion component, the index building component, search component and deletion component are generated, and the source data is imported into the index library based on the index building component to obtain the target search system. Therefore, the present invention reduces the difficulty of search system development and improves development efficiency.

[0134] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a method for establishing a search system provided by an embodiment of the present invention.

[0135] The electronic device 1 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The electronic device 1 can be a computer, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computer sets.

[0136] In this embodiment, the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13 that can be interconnected and communicated via a system bus. The memory 11 stores a search system establishment program 10 that can be executed by the processor 12. Figure 3 Only the electronic device 1 having components 11-13 and the search system establishment program 10 is shown, and those skilled in the art can understand that Figure 3 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0137] The memory 11 includes a memory and at least one type of readable storage medium. The memory provides a cache for the operation of the electronic device 1; the readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1; in other embodiments, the non-volatile storage medium may also be an external storage device of the electronic device 1, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. In this embodiment, the readable storage medium of the memory 11 is generally used to store an operating system and various application software installed on the electronic device 1, such as storing the code of the search system establishment program 10 in an embodiment of the present invention. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.

[0138] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 12 is generally used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the search system establishment program 10.

[0139] The network interface 13 may include a wireless network interface or a wired network interface, and the network interface 13 is used to establish a communication connection between the electronic device 1 and a client (not shown in the figure).

[0140] Optionally, the electronic device 1 may further include a user interface, which may include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0141] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0142] The search system establishment program 10 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 12, it can achieve:

[0143] In response to a search system establishment request issued by a user based on a first client, a table establishment statement template is sent to the first client, and an initial table establishment statement fed back by the first client based on the table establishment statement template is received;

[0144] Parsing the initial table creation statement to obtain the table name and field types of the table header fields of the table to be created, and converting the initial table creation statement into a target table creation statement based on a mapping relationship between the field types of the table creation statement template and the standard field types of the search engine;

[0145] Based on the target table creation statement and the table name, the search system is established to request allocation of an idle search engine server cluster and to establish an index library;

[0146] Obtain program templates corresponding to indexing components, search components and deletion components respectively, generate indexing components, search components and deletion components based on the target table creation statement and the program templates, import source data corresponding to the search system establishment request into the index library based on the indexing components, and obtain the target search system.

[0147] Specifically, the specific implementation method of the processor 12 for the above-mentioned search system establishment program 10 can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0148] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or non-volatile. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0149] The computer-readable storage medium stores a search system establishment program 10, which can be executed by one or more processors. The specific implementation of the computer-readable storage medium of the present invention is basically the same as the above-mentioned embodiments of the search system establishment method, and will not be repeated here.

[0150] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0151] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0153] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0154] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0155] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for establishing a search system, characterized in that: The method comprises: In response to a search system establishment request issued by a user based on a first client, a table establishment statement template is sent to the first client, and an initial table establishment statement fed back by the first client based on the table establishment statement template is received; Parsing the initial table creation statement to obtain the table name and field types of the table header fields of the table to be created, and converting the initial table creation statement into a target table creation statement based on a mapping relationship between the field types of the table creation statement template and the standard field types of the search engine; Based on the target table creation statement and the table name, the search system is established to request allocation of an idle search engine server cluster and to establish an index library; Obtain program templates corresponding to the indexing component, the search component, and the deletion component respectively; Parse the target table creation statement to obtain the table header field name and format of the table to be created, and generate an entity class based on the table header field name and format; Generate an indexing component based on the program template corresponding to the entity class and the indexing component; Extracting a list of field names whose attributes are searchable fields from the target table creation statement, and generating a search component based on the list of field names, the entity class, and a program template corresponding to the search component; Generate a deletion component based on the program template corresponding to the entity class and the deletion component; Based on the indexing component, the source data corresponding to the search system establishment request is imported into the index library to obtain a target search system.

2. The method for establishing a search system according to claim 1, wherein: The method further comprises: When receiving an index library expansion request issued by the user based on the second client, displaying an expansion configuration interface on the display interface of the second client, and receiving a target index library and a field to be sharded input by the user based on the expansion configuration interface; The key value of the field to be sharded is input into a preset calculation formula for calculation, and the data in the target index library is distributed to different nodes of the search engine server cluster based on the calculated value.

3. The method for establishing a search system according to claim 2, wherein: The preset calculation formula is: K%N ==0 (1) K%N ==1 (2) …… K%N==(N-1)(N) Among them, K is the key value of the field to be sharded, and N is the total number of nodes after expansion; Data satisfying formula (1) will be stored in the first node, data satisfying formula (2) will be stored in the second node, and so on, data satisfying formula (N) will be stored in the Nth node.

4. The method for establishing a search system according to claim 1, wherein: The method further comprises: The resource utilization rate of the server to which the search system belongs is obtained in real time or periodically, and when the resource utilization rate exceeds a first threshold, current limiting and degradation processing is performed on the search system.

5. The method for establishing a search system according to claim 1, wherein: The method further comprises: When a data write request from a user based on a third client is received, it is determined whether the amount of data to be written is greater than a second threshold. If it is determined to be greater than the second threshold, an asynchronous write operation is performed on the data to be written.

6. The method for establishing a search system according to claim 1, wherein: The program template corresponding to the index building component includes the API interface name, source data name, field version information and action occurrence time information written into the index library; The program template corresponding to the search component includes sorting parameters, paging parameters, search fields and return fields.

7. A search system establishment device, used to implement the search system establishment method according to any one of claims 1 to 6, characterized in that: The device comprises: A response module, used to respond to a search system establishment request issued by a user based on a first client, send a table establishment statement template to the first client, and receive an initial table establishment statement fed back by the first client based on the table establishment statement template; A parsing module, used to parse the initial table creation statement, obtain the table name of the table to be created and the field type of the table header field, and convert the initial table creation statement into a target table creation statement based on the mapping relationship between the field type of the table creation statement template and the standard field type of the search engine; An allocation module, used to allocate an idle search engine server cluster to the search system based on the target table creation statement and the table name and to establish an index library; A generation module is used to respectively obtain program templates corresponding to indexing components, search components and deletion components, generate indexing components, search components and deletion components based on the target table creation statement and the program templates, and import the source data corresponding to the search system establishment request into the index library based on the indexing component to obtain the target search system.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a search system establishment program that can be executed by the at least one processor, and the search system establishment program is executed by the at least one processor so that the at least one processor can execute the search system establishment method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a search system establishment program, and the search system establishment program can be executed by one or more processors to implement the search system establishment method according to any one of claims 1 to 6.

Images