Search management method and device based on Elasticsearch

Abstract

The invention discloses an Elasticsearch search management method and device, and relates to the technical field of distributed search management, a unified project framework is constructed based on Springboot, an interface and business logic are packaged, authority control, log recording and an exception handling mechanism are combined, standard butt joint with an Elasticsearch server and a data source server is realized, and the search management efficiency is improved. According to the method, the use process of Elasticsearch is more concise, index management is safer and more orderly, the problems of index unauthorized tampering, multi-channel index confusion and index reconstruction required by mapping type change are avoided, and the effect of improving system maintainability, expansibility and service response efficiency is achieved. In this way, the use requirement for Elasticsearch in a complex service scene can be better met, and the operation, maintenance and development cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed search management, and particularly relates to a search management method and device based on Elasticsearch. BACKGROUND

[0002] Elasticsearch, as a distributed search and analysis engine of ElasticStack core, has been widely used in log analysis, full-text search, business data query and other scenarios due to its efficient storage and fast retrieval capability for structured and unstructured data. Although there are various Elasticsearch integration tools (such as related jar packages) in the current industry, such tools are generally imperfect in function, can only support a small part of basic functions of Elasticsearch, lack unified project framework support, and thus it is difficult for developers to extend functions to adapt to complex business requirements in actual projects, increasing the difficulty of connecting and using Elasticsearch.

[0003] At the same time, Elasticsearch officially only provides basic configuration and operation schemes such as index creation and document reading and writing, without covering key practical functions: first, there is no unified permission verification process for all interfaces, and the security risk of index being tampered with by unauthorized operation is easy to occur; second, there is no standardized interface for document addition, deletion and modification, which reduces the security and maintainability of the search server; third, there is no standardized scheme for log recording and exception handling, and it is difficult to track the operation trail during system operation, and it is difficult to quickly locate and handle exceptions, increasing the operation and maintenance cost; in addition, the official scheme does not solve the problems of multi-channel index differentiation management and flexible adjustment of index mapping type, further restricting the stable application of Elasticsearch in complex business scenarios.

[0004] The defects of the above existing technologies make Elasticsearch face problems such as complicated connection process, difficult function expansion, no data security guarantee and low operation efficiency in actual application, and an Elasticsearch search management scheme that can systematically solve these problems is urgently needed. SUMMARY

[0005] In view of the above problems, the present application provides a search management method and device based on Elasticsearch, including the following contents:

[0006] In a first aspect, the present application provides a search management method based on Elasticsearch, which comprises:

[0007] A project is built based on Springboot, dependencies corresponding to the Springboot and Elasticsearch related dependencies are introduced, and a project basic framework is constructed;

[0008] A controller class and a service class are written in the project basic framework, the controller class provides interfaces for index operation, document operation and query condition operation, and the service class is used to implement business logic corresponding to the interfaces;

[0009] An Elasticsearch server address is configured, and the project basic framework is used to realize the docking of the Elasticsearch server;

[0010] A data source server is docked, document data is received through the interface provided by the controller class, and the business logic of the service class is called to write the document data into the index of the Elasticsearch server;

[0011] Based on the project basic framework, JSON network tokens and SpringSecurity are used to realize permission control, and all interfaces are subjected to permission verification;

[0012] A log recording component and a unified exception handling class are configured in the project basic framework, the system running process is logged through the log recording component, and the exception in the system running is captured and handled through the unified exception handling class.

[0013] Optionally, the controller class includes:

[0014] A new index interface is developed, the new index interface receives a request object containing an index name and a mapping configuration, performs parameter verification on the request object, and triggers an index creation process;

[0015] A query document list interface is developed, the query document list interface receives a request object containing query conditions and paging parameters, performs verification on the request object, triggers a document retrieval process, and obtains a paginated document list meeting the conditions from the Elasticsearch server;

[0016] A document data storage interface is developed, the document data storage interface receives a request object containing document content, a target index name and a document ID, triggers a document writing process based on the request object, and writes the document data into the corresponding index;

[0017] A new query condition interface is developed, the new query condition interface receives a request object containing a condition name, a query statement and a module, performs verification on the request object, triggers a query condition saving process, and stores a query condition template.

[0018] Optionally, the service class includes:

[0019] A method for creating an index is developed, which receives a request object containing an index name, index settings, index mapping, and channel code, creates an Elasticsearch index based on the request object, and saves index metadata to a database;

[0020] A method for creating a query condition is developed, which receives a request object containing a condition name, a corresponding index name, and condition configuration, adds a reusable query condition under the specified index based on the request object, and persists the query condition to the database;

[0021] A method for querying a document list is developed, which receives a request object containing an index name, paging information, a search keyword, and a query condition ID, implements full-text search based on the request object, filters out document data that meets the conditions, and returns a paging result;

[0022] A method for storing document data is developed, which receives a request object containing a target index name and document data, writes the document data into the corresponding index based on the request object, and synchronously updates the mapping structure of the corresponding index in the database.

[0023] Optionally, the method further includes:

[0024] A multi-channel configuration management function is added to the business logic of the service class, index metadata is recorded with the corresponding channel code, and different channels are managed differently through the channel code;

[0025] A mapping type dynamic modification function is added to the business logic of the service class, when the index mapping type needs to be changed, the mapping modification process is triggered through the interface of the controller class, the index mapping type is directly modified, and the index does not need to be rebuilt;

[0026] An index-level permission refinement management function is added to the permission control logic, different index operation permissions are assigned based on roles, and different roles are distinguished in terms of operation permissions on the index through the permission verification.

[0027] Optionally, the configuration log recording component and the unified exception handling class include:

[0028] The log configuration file path is specified in the project configuration file, and log output rules such as log output format, output level, and storage location are configured based on the path;

[0029] A log factory is introduced in the Java class, and normal state information and error information in system operation are recorded through a log output method;

[0030] The controller notification annotation and the exception handling annotation define a unified exception handling class, and the system global exception and the business exception are captured through the unified exception handling class, and a standardized error response containing an error code and error information is returned.

[0031] In a second aspect, the application provides a search management device based on Elasticsearch, which comprises:

[0032] A framework building unit is configured to build a project based on Springboot, introduce dependencies corresponding to the Springboot and Elasticsearch related dependencies, and build a project basic framework.

[0033] A code writing unit is configured to write a controller class and a service class in the project basic framework, the controller class provides interfaces for index operation, document operation and query condition operation, and the service class is used to implement business logic corresponding to the interfaces.

[0034] An Elasticsearch connecting unit is configured to configure an Elasticsearch server address and implement connection with the Elasticsearch server based on the project basic framework.

[0035] A data source interaction unit is configured to connect a data source server, receive document data through the interfaces provided by the controller class, and write the document data into the index of the Elasticsearch server by calling the business logic of the service class.

[0036] A permission control unit is configured to implement permission control based on the project basic framework using a JSON network token and SpringSecurity, and perform permission verification on all interfaces.

[0037] A log and exception handling unit is configured to configure a log recording component and a unified exception handling class in the project basic framework, record the system running process through the log recording component, and capture and handle exceptions in the system running process through the unified exception handling class.

[0038] Optionally, the code writing unit writes the controller class, which comprises:

[0039] A new index interface is developed, the new index interface receives a request object containing an index name and a mapping configuration, performs parameter verification on the request object, and triggers an index creation process.

[0040] A query document list interface is developed, which receives a request object containing query conditions and pagination parameters, triggers a document retrieval process after the request object is checked, and obtains a paginated document list meeting the conditions from the Elasticsearch server;

[0041] A document data storage interface is developed, which receives a request object containing document content, a target index name, and a document ID, triggers a document writing process based on the request object, and writes the document data into the corresponding index;

[0042] An added query condition interface is developed, which receives a request object containing a condition name, a query statement, and a module to which the condition belongs, triggers a query condition saving process after the request object is checked, and stores a query condition template.

[0043] Optionally, the code writing unit writes a service class including:

[0044] A create index method is developed, which receives a request object containing an index name, index settings, index mapping, and channel code, creates an Elasticsearch index based on the request object, and saves index metadata to a database;

[0045] A create query condition method is developed, which receives a request object containing a condition name, a target index name, and condition configuration, adds a reusable query condition under the specified index based on the request object, and persists the query condition to a database;

[0046] A query document list method is developed, which receives a request object containing an index name, pagination information, a search keyword, and a query condition ID, implements full-text search based on the request object, filters out document data meeting the conditions, and returns a paginated result;

[0047] A document data storage method is developed, which receives a request object containing a target index name and document data, writes the document data into the corresponding index based on the request object, and synchronously updates the mapping structure of the corresponding index in the database.

[0048] Optionally, the apparatus further includes a function extension unit configured to add a multi-channel configuration management function to the business logic of the service class, record a channel code in the index metadata, and implement differentiated management of indexes by different channels through the channel code.

[0049] A mapping type dynamic modification function is added to the business logic of the service class, a mapping modification process is triggered through an interface of the controller class when an index mapping type needs to be changed, and the index mapping type is directly modified without the need to rebuild the index.

[0050] The index level permission fine management function is added in the permission control logic, different index operation permissions are assigned based on roles, and the operation permission of different roles on the index is distinguished through the permission verification.

[0051] Optionally, the log and exception processing unit is specifically configured to specify a log configuration file path in a configuration file of the project, and configure log output formats, output levels, storage locations and other log output rules based on the path.

[0052] A log factory is introduced in the Java class, and normal state information and error information in system operation are recorded through a log output method.

[0053] The controller notification annotation and the exception processing annotation are used to define a unified exception processing class, global system exceptions and business exceptions are captured through the unified exception processing class, and a standardized error response containing an error code and error information is returned.

[0054] In a third aspect, the present application provides a device, the device comprising a memory and a processor, the memory being configured to store instructions or codes, and the processor being configured to execute the instructions or codes to enable the device to perform the Elasticsearch-based search management method introduced in any of the implementation manners of the first aspect.

[0055] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing codes, and when the codes are executed, a device executing the codes implements the Elasticsearch-based search management method introduced in any of the implementation manners of the first aspect.

[0056] The application provides a search management method based on Elasticsearch. In the execution of the method, first, a project is built based on Springboot, the corresponding dependencies of the Springboot and Elasticsearch related dependencies are introduced, and a project basic framework is constructed. Then, a controller class and a service class are written in the project basic framework. The controller class provides interfaces for index operation, document operation and query condition operation, and the service class is used to implement the business logic corresponding to the interfaces. Then, the address of the Elasticsearch server is configured, the project basic framework is used to realize the docking of the Elasticsearch server, the data source server is docked, the document data is received through the interface provided by the controller class, the business logic of the service class is called to write the document data into the index of the Elasticsearch server, the JSON network token and SpringSecurity are used to realize permission control based on the project basic framework, the permission of all interfaces is checked, and finally, the log recording component and the unified exception handling class are configured in the project basic framework. The system running process is recorded through the log recording component, and the exception in the system running is captured and processed through the unified exception handling class. In this way, by building a unified project framework based on Springboot and encapsulating interfaces and business logic, combined with permission control, log recording and exception handling mechanism, and at the same time realizing the standard docking of the Elasticsearch server and the data source server, the use process of Elasticsearch is more concise, the index management is safer and more orderly, the problems of index unauthorized tampering, multi-channel index confusion and index reconstruction due to mapping type change are avoided, and the effects of improving the maintainability, expansibility and business response efficiency of the system are achieved. In this way, the use demand of Elasticsearch in complex business scenarios can be better adapted, and the operation and development cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0058] Figure 1 A flowchart of a search management method based on Elasticsearch provided by the embodiments of the application;

[0059] Figure 2 A structural schematic diagram of a search management device based on Elasticsearch provided by the embodiments of the application. DETAILED DESCRIPTION

[0060] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0061] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0062] Figure 1 A flowchart of a search management method based on Elasticsearch provided by an embodiment of the present application is shown in FIG. 1. In combination with FIG. 1, the search management method based on Elasticsearch provided by the embodiment of the present application can include the following steps. Figure 1

[0063] S101, build a project based on Springboot, introduce the dependencies corresponding to the Springboot and the Elasticsearch related dependencies, and construct a project basic framework.

[0064] First, carry out the project basic framework construction work, the core is to rely on Springboot to complete the project initialization and dependency integration, and lay a foundation for subsequent function development. Springboot is a rapid development scaffold based on Spring framework, which has the characteristics of automatic configuration, starting dependency, hot deployment and built-in monitoring, can greatly simplify the project building process, and quickly build micro-service applications. In the specific operation, first create a standard project through the Maven tool, define the GroupId as “cn.xxx.springboot” to identify the organization to which the project belongs, the ArtifactId as “xxx-springboot” as the unique project name, the Version as “1.0-SNAPSHOT” to identify the development stage version, and generate a standard project structure containing the src / main / java source code directory and the src / main / resources resource directory, to clearly define the storage path of the code and the configuration file.

[0065] ​Then the dependency configuration is performed. In the dependency configuration file of the project, first, the Springboot parent project is specified and the 2.0.6.RELEASE version is selected. The parent project has uniformly managed the versions of commonly used dependencies (such as SpringMVC and embedded Tomcat server) in the Spring ecosystem. When subsequent dependencies are introduced, the version does not need to be manually specified, which can effectively avoid version conflict problems. Based on the functional requirements of the project, the corresponding core dependencies of Springboot are introduced, including SpringbootWeb dependency. This dependency includes the components required for HTTP request processing and RESTful API development, which can support the development of controller classes and interfaces, and realize the reception and response return of client requests.

[0066] At the same time, the Elasticsearch related dependencies are introduced, including Elasticsearch Java client dependency and Elasticsearch REST client dependency, both of which are version 8.15.5. Elasticsearch is a distributed search and analysis engine that supports efficient storage, indexing, and retrieval of data. The Elasticsearch Java client dependency provides API support for core operations such as index creation and document read-write. The Elasticsearch REST client dependency encapsulates RESTful request logic and can establish a connection with the Elasticsearch server through the HTTP protocol, simplifying the development work of the network communication layer. In addition, considering the subsequent requirements of permission control and log processing, the SpringSecurity dependency and SLF4J related dependency are introduced in advance. The SpringSecurity dependency provides identity authentication and authorization management function support for subsequent permission control. The SLF4J related dependency is a log facade framework that adapts to the underlying log implementation and lays the foundation for log recording functions.

[0067] Through the above operations, a project foundation framework that supports Web service and Elasticsearch interaction can be quickly built. With the version management capability of the Springboot parent project, dependency conflicts can be effectively avoided. Without the need for manual configuration of servers and communication logic, the project initialization period is greatly shortened, providing a stable technical foundation for the development of controller classes, service classes, and function integration.

[0068] S102, controller classes and service classes are written in the project foundation framework. The controller classes provide interfaces for index operations, document operations, and query condition operations. The service classes are used to implement the business logic corresponding to the interfaces.

[0069] After the basic framework is set up, the controller class and the service class need to be developed: the controller class is responsible for providing interfaces for index operation, document operation, and query condition operation, and the service class corresponds to the business logic that implements these interfaces. The two work together to form a complete closed loop of receiving requests, processing logic, and returning results.

[0070] The controller class uses “ / search” as the unified interface prefix and provides four core interfaces. Each interface first checks the legality of the request parameters and then triggers the corresponding service class logic. The specific process is as follows: the index creation interface is set to “ / creatIndex”, uses POST request, receives the request object containing the index name (identifies the Elasticsearch data storage container) and the mapping configuration (defines the field type and storage rules); through the preset verification rules (such as non-empty index name and mapping format compliance), the parameters are verified, and the service class index creation process is triggered after the verification is passed, and a standardized success response is returned successfully.

[0071] The query document list interface is set to “ / getDocumentList”, uses POST request, and receives the request object containing the query condition (such as search keywords) and the paging parameter (page number, number of pages per page); after verifying the legality of the paging parameter (such as page number ≥ 1 and reasonable number of pages per page), the service class document retrieval process is triggered, and the page document list that meets the conditions is obtained from Elasticsearch and returned.

[0072] The document data storage interface is set to “ / putData”, uses POST request, and receives the request object containing the document content (JSON format business data), the target index name (specifies the write index), and the document ID (unique identifier of the document); based on the request parameters, the service class document writing process is triggered, the data is written to the corresponding index, and a response containing the document ID is returned successfully.

[0073] The new query condition interface is set to “ / createCondition”, uses POST request, and receives the request object containing the condition name (identifies the query template), the query statement (such as field weight configuration), and the module (distinguishes business scenarios); after verifying the parameters (such as non-empty condition name), the service class query condition saving process is triggered, and the reusable query condition template is stored.

[0074] The service class interfaces with Elasticsearch and the database to implement the business logic corresponding to the controller interface, mainly including four core methods:

[0075] Index creation method: receive request object containing index name, index settings (such as shard configuration), index mapping, channel code (for multi-channel management); call Elasticsearch client to create index in server, and construct index meta information (containing index name, mapping configuration, channel code) at the same time, store in database through data access layer, realize "index creation-meta information persistence" synchronization.

[0076] Query condition creation method: receive request object containing condition name, index name, condition configuration (such as field weight); first verify the legality of the index (confirm existence by querying database), then build query condition entity, and persist it to database to form reusable query condition under specified index.

[0077] Query document list method: receive request object containing index name, paging information, search keyword, query condition ID; get preset configuration (such as field weight) from database according to query condition ID, combine with keyword to build Elasticsearch full-text search logic, execute search, filter documents meeting conditions, and encapsulate into paging result containing document list, total number and page number, and return.

[0078] Document data storage method: receive request object containing target index name and document data; call Elasticsearch client to write document into corresponding index, and analyze document field at the same time; if there is new field or field type change, update mapping structure of the index in database synchronously, to ensure consistency between Elasticsearch and database meta information.

[0079] S103, configure Elasticsearch server address, and realize connection with the Elasticsearch server based on the project basic framework.

[0080] After completing the development of the controller and service classes, it's necessary to establish a connection between the project and the Elasticsearch server, which is fundamental for subsequent data operations. First, in the configuration file (e.g., application.yml) under the project's src / main / resources directory, add the Elasticsearch server address configuration: for single-node deployment, configure the HTTP address of a single server (default port 9200); for cluster deployment, configure multiple node addresses separated by commas. This way, when switching environments later, only the configuration file needs to be modified, without changing the business logic, making operations more convenient. Next, relying on the Elasticsearch REST client dependency introduced in S101, and leveraging Spring Boot's auto-configuration features, initialize an Elasticsearch client instance. This client will automatically read the server address in the configuration file, establish an HTTP connection, and achieve communication with the Elasticsearch server; if custom connection parameters (such as connection timeout, request timeout) need to be customized, they can be added to the configuration file. Finally, verify the integration effectiveness: call the basic functions of the Elasticsearch client in the service class, such as querying the server health status and checking if test indexes exist. If a server response is received normally (e.g., a health status of "green"), the connection is successful; if it fails, an error message will be triggered. In this case, it is necessary to check whether the server network is working properly and whether the port is open to ensure that subsequent business operations can be executed normally.

[0081] S104. Connect to the data source server, receive document data through the interface provided by the controller class, and call the business logic of the service class to write the document data into the index of the Elasticsearch server.

[0082] After completing the Elasticsearch integration, you can connect to the data source server and write external document data into the Elasticsearch index.

[0083] First, agree on the data interaction specifications with the data source server: the document data format must be consistent with the request object structure of the controller class " / search / putData" interface, including the target index name (specifying the index to be written), document ID (uniquely identifying the document), and document content (JSON format business data, such as insurance product name, code, etc.); at the same time, determine that the interface call method is a POST request, and the response format is a standardized JSON response with status code and prompt information to ensure smooth interaction between the two parties.

[0084] Then the data source server calls the " / search / putData" interface through HTTP request, and transmits the document data conforming to the specification in the request body. After the controller class receives the request, it parses it into the corresponding request object and passes it to the service class.

[0085] Finally, the service class calls the document data storage method, and according to the target index name in the request object, it calls the Elasticsearch client to write the document data into the corresponding index. During the process, the document fields are analyzed, and if there are new fields or field type changes, the mapping structure of the index in the database is updated synchronously; after the operation is completed, the controller class returns the standardized response, and returns the "success" prompt with the 200 status code if successful, or returns the corresponding error information if failed.

[0086] S105, based on the project foundation framework, using JSON network token and Spring Security to realize permission control, and performing permission verification on all interfaces;

[0087] To avoid unauthorized tampering with the index and ensure interface security, after implementing data writing in S104, the Spring Security dependency introduced in S101 needs to be used in combination with JWT (JSON Web Token) to build a permission control system for all interfaces.

[0088] First, create a Security Config configuration class to enable Spring Security features through annotations. In the configuration class, set the interface access rules: all " / search / **" core interfaces defined in S102 require user authentication to access; static resources (such as system icons and style files) are allowed to be accessed anonymously to improve access efficiency. At the same time, configure the password encryption method (such as BCrypt encryption), which can be used for user password storage and verification if the system has a login function. In addition, register the JWT authentication filter in the Spring Security filter chain to ensure that all requests enter the interface first through JWT verification.

[0089] Then implement the JWT authentication logic: when the client requests the interface, it needs to carry the JWT token (format: "Bearer+token string") in the HTTP request header. The JWT filter extracts the token, and if the token does not exist or the format is incorrect, it returns an "Unauthorized" response; if the format is correct, it parses the token through a pre-set key, verifies the signature (anti-tampering) and expiration time (to ensure validity), and returns a "Token Invalid" response if the verification fails, and extracts user role information if the verification is successful.

[0090] Finally, the interface permission verification is performed: the user role and the preset index level permission rule are combined to determine whether the user has the current operation permission (for example, "administrator" can operate all indexes, and "channel operator" can only operate the indexes of the channel to which he belongs). If there is no permission, a "no permission" response is returned, and if there is permission, the request is allowed to enter the interface to perform business logic, realizing permission control of all interfaces.

[0091] In the project base framework, a log recording component and a unified exception handling class are configured, the system running process is logged through the log recording component, and the exception in the system running is captured and handled through the unified exception handling class.

[0092] To facilitate subsequent problem troubleshooting and improve system maintainability, after implementing permission control in S105, a log recording component and a unified exception handling class need to be configured.

[0093] Configure the log recording component: first, specify the path of the log configuration file (such as logback-spring.xml) in the project configuration file, which is stored in the src / main / resources directory. Then define the output rules in the log configuration file: set the output format to "timestamp-thread name-log level-class name-information" to clearly record the log key information; set the output level to record INFO and above logs to cover normal state and error information; use date-based rolling storage, limit a single log file to 100MB, and retain logs within 30 days to avoid occupying too much storage space.

[0094] Finally, in the core methods of the controller class and the service class, record the key operations through the log factory, such as "index creation success" with the index name when the index creation is successful, and "document write failed" with the document ID when the document write fails, to provide a basis for subsequent problem troubleshooting.

[0095] Configure the unified exception handling class: use the controller notification annotation and the log4j annotation to define a unified exception handling class that applies to all controllers, which can capture exceptions in system running, including global exceptions (such as null pointer exceptions and network exceptions) and business exceptions (such as "index does not exist" and "no permission").

[0096] Handle exceptions and respond: capture global exceptions, record detailed stack information, and return a response containing a 500 error code and "server internal error" to avoid exposing underlying exception details; capture business exceptions, record specific exception information, and return a response containing the corresponding error code (such as 404) and a prompt. At the same time, through error code enumeration (such as SUCCESS(200,"success") and UNAUTHORIZED(401,"unauthorized")), ensure the standardization of the response, which is convenient for the client to identify the error type.

[0097] To adapt to complex business scenarios, service class functions need to be extended and further refined permission control:

[0098] Multi-channel configuration management: In the creation index, query index method of S102 service class, perfect channel code logic, create index, channel code (such as "channel_001") into index metadata and store in the database; When querying and modifying the index, filter the index according to the channel code of the current operation to realize the isolated management of indexes of different channels and avoid confusion.

[0099] Dynamic modification of mapping type: Add a mapping modification method in S102 service class, and add " / updateIndexMapping" interface in the controller class. When the index mapping type needs to be changed (such as changing the field from integer type to string type), the client transmits the target index name and new mapping configuration, the service class calls the Elasticsearch client to directly update the mapping, synchronizes the database mapping metadata, and does not need to rebuild the index.

[0100] Index-level permission refinement: Based on S105, a "role-index permission" association table (stored in the database) is added, which records the index range and operation type that different roles (such as "administrator" and "channel operator") can operate. When the client requests the interface, the service class not only verifies the legality of the role, but also queries the index permission list of the role. If there is no permission, an exception is triggered, and the unified exception handling class of S106 returns an error response to avoid unauthorized tampering.

[0101] The above is some specific implementation ways of the search management method based on Elasticsearch provided by the embodiment of the application. Based on this, the application also provides a corresponding device. The device provided by the embodiment of the application will be introduced from the perspective of functional modularization.

[0102] Figure 2 The structure diagram of the search management device based on Elasticsearch provided by the embodiment of the application. In combination with the device provided by the embodiment of the application, the search management device 200 based on Elasticsearch provided by the embodiment of the application comprises: Figure 2

[0103] The framework building unit 210 is configured to build a project based on Springboot, introduce the dependencies corresponding to the Springboot and the Elasticsearch related dependencies, and build a project basic framework;

[0104] ​The code writing unit 220 is configured to write a controller class and a service class in the project base framework, the controller class provides interfaces of index operation, document operation and query condition operation, and the service class is used to implement business logic corresponding to the interfaces;

[0105] The Elasticsearch connecting unit 230 is configured to configure an Elasticsearch server address, and connect the Elasticsearch server based on the project base framework;

[0106] The data source interacting unit 240 is configured to connect a data source server, receive document data through the interfaces provided by the controller class, and write the document data into an index of the Elasticsearch server by calling the business logic of the service class;

[0107] The permission control unit 250 is configured to implement permission control by using a JSON network token and SpringSecurity based on the project base framework, and perform permission verification on all interfaces;

[0108] The log and exception processing unit 260 is configured to configure a log recording component and a unified exception processing class in the project base framework, record system running process through the log recording component, and capture and process exceptions in system running through the unified exception processing class.

[0109] In an implementation manner of the embodiment of the application, the code writing unit writes the controller class to include:

[0110] An index adding interface is developed, the index adding interface receives a request object containing an index name and a mapping configuration, performs parameter verification on the request object, and triggers an index creation process;

[0111] A query document list interface is developed, the query document list interface receives a request object containing query conditions and paging parameters, performs verification on the request object, triggers a document retrieval process, and obtains a paging document list meeting the conditions from the Elasticsearch server;

[0112] A document data storage interface is developed, the document data storage interface receives a request object containing document content, a target index name and a document ID, triggers a document writing process based on the request object, and writes the document data into a corresponding index;

[0113] An added query condition interface is developed, the added query condition interface receives a request object containing a condition name, a query statement and a module, performs verification on the request object, triggers a query condition saving process, and stores a query condition template.

[0114] In an implementation form of the embodiment of the application, the code writing unit writes the service class, which comprises:

[0115] A method for creating an index is developed, which receives a request object containing an index name, index settings, index mapping and channel code, creates an Elasticsearch index based on the request object, and saves index meta information to a database;

[0116] A method for creating a query condition is developed, which receives a request object containing a condition name, a corresponding index name and condition configuration, adds a reusable query condition under a specified index based on the request object, and persists the query condition to a database;

[0117] A method for querying a document list is developed, which receives a request object containing an index name, paging information, a search keyword and a query condition ID, implements full-text search based on the request object, filters out document data meeting the conditions and returns a paging result;

[0118] A method for storing document data is developed, which receives a request object containing a target index name and document data, writes the document data into a corresponding index based on the request object, and synchronously updates the mapping structure of the corresponding index in the database.

[0119] In an implementation form of the embodiment of the application, the device further comprises a function extension unit, configured to add a multi-channel configuration management function in the business logic of the service class, record a corresponding channel code in index meta information, and realize differentiated management of indexes by different channels through the channel code;

[0120] A mapping type dynamic modification function is added in the business logic of the service class, when an index mapping type needs to be changed, a mapping modification process is triggered through an interface of the controller class, the index mapping type is directly modified, and the index does not need to be rebuilt;

[0121] An index level permission fine management function is added in the permission control logic, different index operation permissions are assigned based on roles, and operation permission differentiation of the index by different roles is realized through the permission verification.

[0122] In an implementation form of the embodiment of the application, the log and exception processing unit is specifically configured to specify a log configuration file path in a configuration file of a project, configure log output rules such as a log output format, an output level and a storage location based on the path;

[0123] A log factory is introduced in a Java class, and normal state information and error information in system running are recorded through a log output method;

[0124] The controller notification annotation and the exception handling annotation definition unified exception handling class, the system global exception and the business exception are captured through the unified exception handling class, and the standardized error response containing the error code and the error information is returned.

[0125] The application further provides a corresponding device and a computer storage medium for implementing the scheme provided by the application.

[0126] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes to enable the device to execute the method described in any embodiment of the application.

[0127] The computer storage medium stores codes, and when the codes are executed, the device executing the codes implements the method described in any embodiment of the application.

[0128] From the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps of the above-mentioned embodiment methods can be implemented by means of software plus a general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a read-only memory (English: read-only memory, ROM) / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the method described in each embodiment or some parts of the embodiments of the present application.

[0129] It can be understood that in the specific embodiments of the present application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved need to obtain user permission or consent when the above embodiments of the present application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0130] It should be noted that, in the specification, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0131] It should also be noted that each of the embodiments described in the specification illustrate aspects of the application and are not meant to be an exhaustive list of all possible embodiments. The scope of the application is therefore intended to cover all possible embodiments that are within the scope of the claims. Furthermore, the description of the embodiments is not meant to limit the scope of the application to the exact embodiments described. The scope of the application is intended to cover all possible embodiments that are within the scope of the claims.

[0132] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed by the present application can be easily thought of by those skilled in the art without any creative effort, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for Elasticsearch-based search management, characterized by, The method comprises: Based on Springboot project, introduce the corresponding dependencies of Springboot and Elasticsearch related dependencies, build project basic framework; Write controller class and service class in the project basic framework, the controller class provides the interface of index operation, document operation and query condition operation, the service class is used to realize the business logic corresponding to the interface; Configure the address of the Elasticsearch server, and realize the docking of the Elasticsearch server based on the project basic framework; Dock the data source server, receive the document data through the interface provided by the controller class, call the business logic of the service class to write the document data into the index of the Elasticsearch server; Based on the project basic framework, use JSON network token and SpringSecurity to realize permission control, and perform permission verification on all interfaces; Configure the log recording component and unified exception handling class in the project basic framework, record the system running process through the log recording component, and capture and process the exception in the system running through the unified exception handling class.

2. The method of claim 1, wherein, The controller class comprises: Develop a new index interface, the new index interface receives a request object containing an index name and a mapping configuration, performs parameter verification on the request object, and triggers an index creation process; Develop a query document list interface, the query document list interface receives a request object containing query conditions and paging parameters, performs verification on the request object, triggers a document retrieval process, and obtains a paginated document list that meets the conditions from the Elasticsearch server; Develop a document data storage interface, the document data storage interface receives a request object containing document content, target index name and document ID, triggers a document writing process based on the request object, and writes the document data into the corresponding index; Develop a new query condition interface, the new query condition interface receives a request object containing condition name, query statement and module, performs verification on the request object, triggers a query condition saving process, and stores the query condition template.

3. The method of claim 1, wherein, The service class comprises: Develop a create index method, the create index method receives a request object containing an index name, index settings, index mapping and channel code, creates an Elasticsearch index based on the request object, and saves index metadata to the database; Develop a create query condition method, the create query condition method receives a request object containing condition name, index name and condition configuration, adds reusable query conditions under the specified index based on the request object, and persists the query conditions to the database; Develop a query document list method, the query document list method receives a request object containing index name, paging information, search keywords and query condition ID, implements full-text search based on the request object, filters out the document data that meets the conditions, and returns the paging result; A method for storing document data is provided, which receives a request object containing a target index name and document data, writes the document data into a corresponding index based on the request object, and synchronously updates a mapping structure of the corresponding index in a database.

4. The method of claim 1, wherein, The method further comprises: A multi-channel configuration management function is added to the service logic of the service class, the channel code is recorded in the index meta information, and the channel code is used to distinguish and manage the index in different channels; A mapping type dynamic modification function is added to the service logic of the service class, when the index mapping type needs to be changed, the mapping modification process is triggered through the interface of the controller class, and the index mapping type is directly modified without rebuilding the index; An index-level permission refinement management function is added to the permission control logic, different index operation permissions are assigned based on roles, and the operation permission of the index is distinguished by different roles through the permission check.

5. The method of claim 1, wherein, The configuration log recording component and the unified exception handling class comprise: The log configuration file path is specified in the configuration file of the project, and the log output format, output level, storage location and other log output rules are configured based on the path; A log factory is introduced in the Java class, and normal state information and error information in the system running are recorded through the log output method; A controller notification annotation and an exception handling annotation are used to define a unified exception handling class, the system global exception and the business exception are captured through the unified exception handling class, and a standardized error response containing an error code and error information is returned.

6. An Elasticsearch-based search management apparatus, characterized by comprising: The device comprises: A framework building unit is configured to build a project based on Springboot, introduce dependencies corresponding to the Springboot and Elasticsearch related dependencies, and build a project basic framework; A code writing unit is configured to write a controller class and a service class in the project basic framework, the controller class provides interfaces for index operation, document operation and query condition operation, and the service class is used to implement business logic corresponding to the interfaces; An Elasticsearch connecting unit is configured to configure an Elasticsearch server address and implement connection with the Elasticsearch server based on the project basic framework; A data source interaction unit is configured to connect to a data source server, receive document data through the interfaces provided by the controller class, and call the business logic of the service class to write the document data into the index of the Elasticsearch server; A permission control unit is configured to implement permission control based on the project basic framework using a JSON network token and SpringSecurity, and perform permission check on all interfaces; A log and exception handling unit is configured to configure a log recording component and a unified exception handling class in the project basic framework, record the system running process through the log recording component, and capture and handle exceptions in the system running through the unified exception handling class.

7. The apparatus of claim 6, wherein, The code writing unit writes a controller class, which comprises: Develop an index adding interface, which receives a request object containing an index name and a mapping configuration, triggers an index creation process after parameter checking the request object; Develop a query document list interface, which receives a request object containing query conditions and pagination parameters, triggers a document retrieval process after checking the request object, and obtains a paginated document list that meets the conditions from the Elasticsearch server; Develop a document data storage interface, which receives a request object containing document content, target index name and document ID, triggers a document writing process based on the request object, and writes the document data to the corresponding index; Develop an add query condition interface, which receives a request object containing condition name, query statement and module, triggers a query condition saving process after checking the request object, and stores the query condition template.

8. The apparatus of claim 6, wherein, The code writing unit writes service class including: Develop a create index method, which receives a request object containing index name, index settings, index mapping and channel code, creates an Elasticsearch index based on the request object, and saves index meta information to the database; Develop a create query condition method, which receives a request object containing condition name, index name and condition configuration, adds reusable query conditions under the specified index based on the request object, and persists the query conditions to the database; Develop a query document list method, which receives a request object containing index name, pagination information, search keywords and query condition ID, implements full-text search based on the request object, filters out the document data that meets the conditions and returns the pagination result; Develop a document data storage method, which receives a request object containing target index name and document data, writes the document data to the corresponding index based on the request object, and synchronously updates the mapping structure of the corresponding index in the database.

9. A computing device, comprising: The computing device includes a memory, a processor; The memory is used to store computer programs; The processor is used to execute the computer programs to realize the steps of the method of any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer programs, which are executed by the processor to realize the steps of the method of any one of claims 1 to 5.

Images