Water conservancy data retrieval method and device of distributed double-engine architecture and medium

Abstract

The invention discloses a water conservancy data retrieval method and device of a distributed double-engine architecture and a medium, and relates to the technical field of geographic information retrieval. The method comprises the steps of receiving water conservancy spatial data; encoding and splitting the line data and the surface data to obtain a first data set and a second data set, and adding a position identifier, a type identifier and a unique identifier to the first data set and the second data set; based on a double-engine architecture comprising a full-text retrieval engine and a spatial database, importing the first data set into the full-text retrieval engine to obtain a first data table, and importing the second data set into the spatial database to obtain a second data table; obtaining a relation mapping table representing a mapping relation between the first data table and the second data table according to the position identifier; when a query is responded, an initial result set is retrieved through a full-text retrieval engine. According to the method, the query speed, the composite query precision and the hardware resource utilization rate are all improved.

Description

Technical Field

[0001] The present application relates to the field of geographic information retrieval technology, and in particular to a water conservancy data retrieval method, device and medium with a distributed dual-engine architecture. Background Art

[0002] Innovation in information technology applications is a major national strategic deployment. As a key industry related to people's livelihood, water conservancy must strictly follow the requirements of information technology innovation policies and use software and hardware products that meet information technology innovation standards.

[0003] This addresses two major pain points in spatial data management within the water conservancy industry under the trusted computing environment: traditional relational databases have weak full-text search capabilities and complex operations, making them incapable of supporting terabyte-level data within a watershed. Open-source full-text search engines like ElasticSearch lack strong support for commonly used formats in the water conservancy sector, such as GeoJSON and Shapefile, and storing large amounts of spatial data results in slower full-text search speeds and reduced spatial data accuracy. Furthermore, the trusted computing environment requires compatibility with a domestically developed software and hardware ecosystem, further increasing technical complexity.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows: The existing water conservancy data retrieval technology cannot perform efficient full-text retrieval and precise spatial query, and cannot guarantee cross-engine data consistency and compatibility with domestic platforms. Summary of the Invention

[0005] The embodiments of the present application provide a water conservancy data retrieval method, device and medium with a distributed dual-engine architecture, which can solve the problems of water conservancy data retrieval in the existing technology, such as the inability to conduct efficient full-text retrieval and accurate spatial query, and the inability to ensure cross-engine data consistency and compatibility with domestic platforms.

[0006] In the first aspect, an embodiment of the present application provides a water conservancy data retrieval method with a distributed dual-engine architecture, the method comprising: receiving water conservancy spatial data, the water conservancy spatial data including line data, surface data and point data; encoding and splitting the line data and surface data to obtain a first data set and a second data set, the first data set including attribute information, the second data set including attribute information and spatial information, adding a location identifier, a type identifier and a unique identifier to the first data set and the second data set; based on the dual-engine architecture including a full-text retrieval engine and a spatial database, the first data set is imported into the full-text retrieval engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; according to the location identifier, a relationship mapping table representing the mapping relationship between the first data table and the second data table is obtained; when responding to a query, an initial result set is obtained through a full-text retrieval engine search.

[0007] In one implementation of the present application, after obtaining an initial result set through a full-text search engine search, the method further includes: splitting the initial search result set into a point data subset, a line data subset, and a surface data subset based on a type identifier; querying the corresponding second data table through a relationship mapping table according to the location identifiers of the line data subset and the surface data subset; batch querying the spatial information corresponding to the second data table according to the unique identifier; merging the initial result set and the spatial information, sorting them by the sorting field, and outputting the query results.

[0008] In one implementation of the present application, after receiving water conservancy spatial data, which includes line data, surface data, and point data, the method also includes: splitting the point data into a third data set, adding a location identifier and a unique identifier to the third data set, the third data set including attribute information and spatial information; importing the third data set into a full-text retrieval engine, and creating a corresponding spatial data table in a spatial database to import the third data set.

[0009] In one implementation of the present application, a first data set is imported into a full-text search engine to obtain a first data table, and a second data set is imported into a spatial database to obtain a second data table, specifically including: adopting a two-phase commit protocol, sending a pre-commit request to the spatial database and the full-text search engine in the pre-commit phase; the spatial database temporarily stores the data to be written in a temporary table and locks the resources; the full-text search engine constructs a draft index to be updated and pre-occupies a version number.

[0010] In one implementation of the present application, the method further includes: in the submission phase, if all participants successfully respond to the pre-submission, a submission instruction is sent to all participants: the spatial database migrates the temporary table to the formal table and releases the resource lock; the full-text search engine publishes the index of the pre-occupied version number.

[0011] In one implementation of the present application, the method further includes: implementing a circuit breaker mechanism in the pre-commitment phase, and immediately triggering a rollback if the response times out; in the submission phase, if any participant's response times out or fails, a rollback instruction is sent to all participants and the rollback operation is executed.

[0012] In one implementation of the present application, the method further includes: adding a sorting field to the initial result set, where the sorting field is used to record the original order; and using a universal inverted index for the position identifier and the unique identifier.

[0013] In one implementation of the present application, the method further includes: disabling indexing for non-search fields in a full-text search engine; and setting a unified alias for fields representing space and feature names.

[0014] In the second aspect, an embodiment of the present application also provides a water conservancy data retrieval device with a distributed dual-engine architecture, the device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: receive water conservancy spatial data, the water conservancy spatial data including line data, surface data and point data; encode and split the line data and surface data to obtain a first data set and a second data set, the first data set including attribute information, the second data set including attribute information and spatial information, and add a location identifier, a type identifier and a unique identifier to the first data set and the second data set; based on the dual-engine architecture including a full-text retrieval engine and a spatial database, the first data set is imported into the full-text retrieval engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; based on the location identifier, a relationship mapping table representing the mapping relationship between the first data table and the second data table is obtained; when responding to a query, an initial result set is obtained through a full-text retrieval engine search.

[0015] In a third aspect, an embodiment of the present application also provides a non-volatile computer storage medium for water conservancy data retrieval with a distributed dual-engine architecture, which stores computer executable instructions, and the computer executable instructions are set to: receive water conservancy spatial data, which includes line data, surface data and point data; encode and split the line data and surface data to obtain a first data set and a second data set, the first data set includes attribute information, and the second data set includes attribute information and spatial information, and add a location identifier, a type identifier and a unique identifier to the first data set and the second data set; based on the dual-engine architecture including a full-text retrieval engine and a spatial database, the first data set is imported into the full-text retrieval engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; based on the location identifier, a relationship mapping table representing the mapping relationship between the first data table and the second data table is obtained; when responding to a query, an initial result set is obtained through a full-text retrieval engine search.

[0016] The embodiments of the present application provide a water conservancy data retrieval method, device and medium with a distributed dual-engine architecture. Through the dual-engine collaborative architecture, Hangao Database processes spatial relationship operations and EasySearch processes full-text retrieval, solving the performance bottleneck of a single engine and improving the response speed of basin-level TB data composite queries; an enhanced two-phase commit protocol, pre-commit lock resources and circuit breaker rollback mechanism improve the success rate of cross-system transaction submissions and reduce the average lock waiting time; the dual-engine adaptation layer is optimized for domestic CPU instruction sets, and the index construction efficiency is improved on the Feiteng / Kunpeng platform to meet the requirements of the information innovation policy; a storage strategy for spatial data and attribute data separation, line / surface data split storage, and dual-engine redundancy of point data avoid the loss of spatial data accuracy of the open source engine and reduce the error rate of geometric operations; the GEOID global identifier and relationship mapping table design reduce the complexity of cross-engine data association; a unified alias mechanism (geo_name) reduces index management costs.

[0017] This technology can be widely used in the construction of full-text retrieval systems for water conservancy spatial data under the information and communication technology innovation environment. The distributed dual-engine architecture solves the limitations of a single index structure in complex query scenarios, thereby improving query speed, compound query accuracy, and hardware resource utilization. During development, the differences in instruction sets between domestic CPUs and open source components were taken into consideration, and certain adaptation optimizations were made, which greatly improved the operating efficiency under the information and communication technology innovation environment. The enhanced two-phase commit protocol method optimizes the transaction submission success rate and the average lock waiting time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flow chart of a water conservancy data retrieval method using a distributed dual-engine architecture provided in an embodiment of the present application; Figure 2 A two-phase submission diagram of a water conservancy data retrieval method with a distributed dual-engine architecture provided in an embodiment of the present application; Figure 3 A schematic diagram of the internal structure of a water conservancy data retrieval device with a distributed dual-engine architecture provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] The embodiments of the present application provide a water conservancy data retrieval method, device and medium with a distributed dual-engine architecture, which solves the problems of water conservancy data retrieval in the prior art, such as the inability to conduct efficient full-text retrieval and accurate spatial query, and the inability to ensure cross-engine data consistency and compatibility with domestic platforms.

[0021] The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0022] Figure 1 This is a flow chart of a water conservancy data retrieval method using a distributed dual-engine architecture provided in an embodiment of the present application. Figure 1 shown.

[0023] First of all, in the embodiments of the present application, it can be understood that EasySearch is a distributed search database provided by INFINI Labs, which aims to meet the needs of enterprises in various scenarios such as massive unstructured data retrieval, full-text retrieval, vector retrieval, geographic location information query, combined indexing, multilingual support, and aggregate analysis. It supports a variety of domestic hardware technology routes; Hangao Database is an object-relational database with very rich basic database functions, covering the core features of all mainstream databases, and can meet the basic needs of enterprise-level applications. It uses PostgreSQL kernel technology and has high support for spatial data; GIN index is Generalized Inverted Index, a general inverted index, which is an index structure that stores a set of (key, posting list) pairs, which is particularly suitable for full-text search and multi-value correspondence queries.

[0024] In the embodiment of this application, focusing on the water conservancy spatial data management needs in the information and innovation environment, a hybrid search architecture integrating EasySearch and Hangao database is proposed. The main steps include: data processing, data synchronization into the warehouse, data index establishment, and data retrieval. Data processing is not the main content of this application and will not be elaborated on. This application focuses on innovative design and software development for data synchronization into the warehouse and update, index establishment and data retrieval. This application uses the Java development language and SpringBoot framework to develop the processing program, and uses the VUE front-end framework to develop a set of visual data import pages, supporting spatial data files in various formats such as GeoJson, Shapefile, and CAD.

[0025] The embodiment of the present application provides a distributed dual-engine architecture water conservancy data retrieval method, which specifically includes the following steps: Step 10: Receive water conservancy spatial data, which includes line data, surface data, and point data; Step 20: Encode and split the line data and the surface data to obtain a first dataset and a second dataset. The first dataset includes attribute information, and the second dataset includes attribute information and spatial information. Add a location identifier, a type identifier, and a unique identifier to the first dataset and the second dataset. In this step, when the data uploaded by the front-end is line data or surface data, taking the watershed surface data as an example, the processing program receives the uploaded watershed spatial data and processes the data into two arrays: pure attribute data, recorded as array A (i.e., the first data set) and attribute + spatial data, recorded as array B (i.e., the second data set). The table name of the spatial data, area_liuyu, is converted into Base64 encoding, which can be recorded as encoding A.

[0026] Furthermore, each data in array A and array B is inserted into the geo_table field to store the code A and the data attribute mark field geo_type (the value is line or area). Array A and array B must ensure that there is a unique code field GEOID to ensure one-to-one correspondence between the data.

[0027] Step 30: Based on the dual-engine architecture including the full-text search engine and the spatial database, the first data set is imported into the full-text search engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; In this step, array A is imported into the index established in the EasySearch engine, a new table area_liuyu is created in the Hangao database, and array B is stored in the table.

[0028] It is worth noting that when data is synchronized into or updated, the transaction consistency in the two databases must be guaranteed. Because this application is mainly aimed at the fixed attributes of water conservancy spatial data, it is only necessary to consider the atomicity of cross-system operations, and there is no need to consider concurrent modification conflicts. Mainly, when spatial data is written to the Hangao database, the EasySearch index needs to be updated synchronously to avoid the situation where one party succeeds and the other party fails, such as network interruption, which leads to permanent misalignment of spatial data and attribute indexes. An enhanced two-phase commit protocol (2PC) method is designed to solve this problem. First, the entire transaction processing is initiated by the coordinator class Enhanced2PCCoordinator, and executeTransaction is used as the entry method for transaction execution. Next, the process is divided into two stages.

[0029] As an optional embodiment, Figure 2 As shown, the first data set is imported into the full-text search engine to obtain the first data table, and the second data set is imported into the spatial database to obtain the second data table. Specifically, the following steps may be included: Step 301: Using a two-phase commit protocol, a pre-commit request is sent to the spatial database and the full-text search engine in the pre-commit phase; Step 302: The spatial database temporarily stores the data to be written in a temporary table and locks the resources; Step 303: The full-text search engine constructs a draft index to be updated and pre-occupies a version number.

[0030] In this step, during the pre-commit phase, the coordinator traverses all participants, such as Hangao Database and EasySearch, and sends pre-commit requests. During the pre-commit phase, Hangao Database needs to write data to a temporary table and lock the relevant grid, while EasySearch needs to build an index draft and pre-occupy the version number.

[0031] As an optional embodiment, the method may also include: Step 304: In the submission phase, if all participants successfully respond to the pre-submission, a submission instruction is sent to all participants: Step 305: The spatial database migrates the temporary table to the formal table and releases the resource lock; Step 306: The full-text search engine publishes an index of the pre-occupied version number.

[0032] Next, the commit decision phase begins. During the commit phase, Hangao migrates the temporary data to the official table and releases the locks, and EasySearch publishes the index version. If all participants successfully pre-commit, the coordinator sends the commit command; otherwise, a rollback is triggered.

[0033] As an optional embodiment, the method may further include: implementing a circuit breaker mechanism in the pre-commitment phase, and immediately triggering a rollback if the response times out; in the commit phase, if any participant's response times out or fails, sending a rollback instruction to all participants and executing the rollback operation.

[0034] In this step, the two-phase commit protocol (2PC) method has two branches: successful commit and rollback. The specific operations of commit and rollback include updating the database and index, releasing locks, etc.

[0035] Furthermore, a timeout and circuit breaker mechanism was added to the pre-commit phase, using Hystrix to implement resource isolation and circuit breaking to prevent the spread of single-point failures. If any participant does not respond within 300ms, a rollback will be triggered. This is the core mechanism for solving the atomicity problem of cross-system (Hangao Database and EasySearch) operations.

[0036] Step 40: Obtaining a relationship mapping table representing a mapping relationship between the first data table and the second data table according to the location identifier; In this step, a new table table_relationship is created with fields such as code64 and table_name. Code A and the table name are combined into one piece of data and inserted into the table.

[0037] Step 50: In response to the query, an initial result set is retrieved through a full-text search engine.

[0038] In this step, the background program provides a data query interface. When the user calls the interface for query, the query content object is passed using the POST method. This interface calls the EasySearch query interface according to the query requirements, and uses the EasySearch full-text search capability to obtain the first round of search data, converts it into an array format, and records it as array D. The sort field sort is added to array D to record the current data order.

[0039] As an optional embodiment, after obtaining the initial result set through the full-text search engine search, the method may further include: Step 60: splitting the initial search result set into a point data subset, a line data subset, and a surface data subset based on the type identifier; In this step, the data whose data attribute mark field geo_type value is point is filtered out and recorded as array E. The other data are line data and surface data arrays, recorded as array F.

[0040] Step 70: Query the corresponding second data table through the relationship mapping table according to the location identifiers of the line data subset and the surface data subset; In this step, array F is further grouped into F1, F2, F3, etc. according to the geo_table field, and the geo_table collection is obtained. The table_relationship table is queried using the SQL statement in method to obtain the corresponding table name array, which is recorded as array G.

[0041] Step 80: Based on the unique identifier, batch query the spatial information corresponding to the second data table; In this step, for each table name element in array G, the F1, F2, F3... arrays are respectively queried using the GEOID field using the SQL statement in method to obtain the spatial data in the corresponding spatial data table, and a new array set H1, H2, H3... is obtained, and the H series data is assigned according to the sort field in the F series data.

[0042] Step 90: Merge the initial result set and the spatial information, sort by the sorting field, and output the query results.

[0043] Combine these sets with array E to get the final query result, recorded as array Z. Sort the data in array Z in ascending order of the sort field and return the data through the interface.

[0044] As an optional embodiment, after receiving water conservancy spatial data, which includes line data, surface data and point data, the method may also include: splitting the point data into a third data set, adding a location identifier and a unique identifier to the third data set, the third data set including attribute information and spatial information; importing the third data set into a full-text retrieval engine, and creating a corresponding spatial data table in a spatial database to import the third data set.

[0045] In this step, when the data uploaded by the front-end is point data, the data is directly processed into attribute + spatial data, recorded as array C (i.e. the third data set), and the data attribute mark field geo_type is inserted with the value point, and it is ensured that there is a unique coding field GEOID. Array C is imported into the index of EasySearch, and a new table is created in the Hangao database, and data C is imported into the table.

[0046] As an optional embodiment, the method may further include: adding a sorting field to the initial result set, where the sorting field is used to record the original order; and using a universal inverted index for the position identifier and the unique identifier.

[0047] In this step, we add a sort field to the data initially queried by EasySearch to ensure consistent ordering of data returned in subsequent joint queries. We also create indexes in the Hangao database, setting the code64 field of the table_relationship table and the ID field of the spatial data table as GIN indexes. In the Hangao database, we build a GIN index for spatial data based on the GEOID field, enabling efficient multi-value queries.

[0048] As an optional embodiment, the method may further include: disabling indexing for non-search fields in the full-text search engine; and setting a unified alias for fields representing space and feature names.

[0049] In this step, set index: false for fields that do not need to be searched, such as date and phone number, to reduce index overhead. Set a unified alias of geo_name for fields representing spatial feature names to simplify index management and query.

[0050] In summary, point data stores attribute spatial data in the dual engines, while line and surface data stores attribute data in EasySearch and the Hangao database. The geo_table field is designed to store the Base64 encoding of the table name, the geo_type field stores the data type, the GEOID field stores the data's unique code, and the table_relationship table is designed to store the table name and code correspondence table. The dual-engine collaborative query method first performs a full-text attribute search through EasySearch, directly assembling the point data. Line and surface data use the geo_table field to find the corresponding data table in the Hangao database, and then query the attribute spatial data. The data is then assembled and returned as query results.

[0051] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides a water conservancy data retrieval device with a distributed dual-engine architecture, whose structure is as follows: Figure 3 shown.

[0052] Figure 3 This is a schematic diagram of the internal structure of a water conservancy data retrieval device with a distributed dual-engine architecture provided in an embodiment of the present application. Figure 3 As shown, the equipment includes: at least one processor 01; and a memory 02 communicatively connected to the at least one processor; Among them, the memory 02 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 01 so that the at least one processor 01 can: receive water conservancy spatial data, which includes line data, surface data and point data; encode and split the line data and surface data to obtain a first data set and a second data set, the first data set includes attribute information, and the second data set includes attribute information and spatial information, and add a location identifier, a type identifier and a unique identifier to the first data set and the second data set; based on a dual-engine architecture including a full-text retrieval engine and a spatial database, import the first data set into the full-text retrieval engine to obtain a first data table, and import the second data set into the spatial database to obtain a second data table; according to the location identifier, obtain a relationship mapping table representing the mapping relationship between the first data table and the second data table; when responding to a query, obtain an initial result set through a full-text retrieval engine search.

[0053] Some embodiments of the present application provide corresponding Figure 1A non-volatile computer storage medium for water conservancy data retrieval with a distributed dual-engine architecture stores computer-executable instructions, wherein the computer-executable instructions are configured to: receive water conservancy spatial data, which includes line data, surface data, and point data; encode and split the line data and surface data to obtain a first data set and a second data set, wherein the first data set includes attribute information and the second data set includes attribute information and spatial information, and add a location identifier, a type identifier, and a unique identifier to the first data set and the second data set; based on the dual-engine architecture including a full-text search engine and a spatial database, the first data set is imported into the full-text search engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; based on the location identifier, a relationship mapping table representing the mapping relationship between the first data table and the second data table is obtained; and when responding to a query, an initial result set is obtained through a search with the full-text search engine.

[0054] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the IoT device and media embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0055] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0056] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0060] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0061] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0062] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0063] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0064] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A water conservancy data retrieval method based on a distributed dual-engine architecture, characterized in that: The method comprises: Receiving water conservancy spatial data, wherein the water conservancy spatial data includes line data, surface data, and point data; Encoding and splitting the line data and the surface data to obtain a first data set and a second data set, adding a location identifier, a type identifier, and a unique identifier to the first data set and the second data set, wherein the first data set includes attribute information and the second data set includes attribute information and spatial information; Based on the dual-engine architecture including a full-text search engine and a spatial database, the first data set is imported into the full-text search engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; Obtaining, according to the location identifier, a relationship mapping table representing a mapping relationship between the first data table and the second data table; In response to a query, an initial result set is retrieved through the full-text search engine.

2. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 1 is characterized in that: After obtaining an initial result set through the full-text search engine search, the method further includes: Based on the type identifier, the initial search result set is divided into a point data subset, a line data subset, and a surface data subset; According to the position identifiers of the line data subset and the surface data subset, querying the corresponding second data table through the relationship mapping table; According to the unique identifier, batch query the spatial information corresponding to the second data table; The initial result set and spatial information are merged, sorted by the sorting field, and then the query result is output.

3. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 1 is characterized in that: After receiving the water conservancy spatial data, the water conservancy spatial data including line data, surface data and point data, the method further includes: Splitting the point data into a third data set, adding the location identifier and the unique identifier to the third data set, wherein the third data set includes attribute information and spatial information; The third data set is imported into the full-text search engine, and a corresponding spatial data table is created in the spatial database to import the third data set.

4. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 1 is characterized in that: Importing the first data set into the full-text search engine to obtain a first data table, and importing the second data set into the spatial database to obtain a second data table, specifically including: Using a two-phase commit protocol, in the pre-commit phase, a pre-commit request is sent to the spatial database and the full-text search engine; The spatial database temporarily stores the data to be written into a temporary table and locks the resources; The full-text search engine constructs a draft index to be updated and reserves a version number.

5. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 4 is characterized in that: The method further comprises: In the submission phase, if all participants successfully respond to the pre-submission, a submission instruction is sent to all participants: The spatial database migrates the temporary table to a formal table and releases the resource lock; The full-text search engine publishes an index of the pre-occupied version number.

6. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 5 is characterized in that: The method further comprises: Implement a circuit breaker mechanism in the pre-commit phase, and trigger a rollback immediately if the response times out; During the commit phase, if any participant's response times out or fails, a rollback instruction is sent to all participants and a rollback operation is performed.

7. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 2 is characterized in that: The method further comprises: Adding the sorting field to the initial result set, where the sorting field is used to record the original order; A universal inverted index is used for the position identifier and the unique identifier.

8. The water conservancy data retrieval method based on a distributed dual-engine architecture according to claim 1 is characterized in that: The method further comprises: In the full-text search engine, disabling indexing for non-search fields; Set a unified alias for the fields representing space and feature names.

9. A water conservancy data retrieval device with a distributed dual-engine architecture, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Receiving water conservancy spatial data, wherein the water conservancy spatial data includes line data, surface data, and point data; Encoding and splitting the line data and the surface data to obtain a first data set and a second data set, wherein the first data set includes attribute information and the second data set includes attribute information and spatial information, and adding a location identifier, a type identifier, and a unique identifier to the first data set and the second data set; Based on the dual-engine architecture including a full-text search engine and a spatial database, the first data set is imported into the full-text search engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; Obtaining, according to the location identifier, a relationship mapping table representing a mapping relationship between the first data table and the second data table; In response to a query, an initial result set is retrieved through the full-text search engine.

10. A non-volatile computer storage medium for water conservancy data retrieval with a distributed dual-engine architecture, storing computer-executable instructions, characterized in that: The computer executable instructions are configured to: Receiving water conservancy spatial data, wherein the water conservancy spatial data includes line data, surface data, and point data; Encoding and splitting the line data and the surface data to obtain a first data set and a second data set, wherein the first data set includes attribute information and the second data set includes attribute information and spatial information, and adding a location identifier, a type identifier, and a unique identifier to the first data set and the second data set; Based on the dual-engine architecture including a full-text search engine and a spatial database, the first data set is imported into the full-text search engine to obtain a first data table, and the second data set is imported into the spatial database to obtain a second data table; Obtaining, according to the location identifier, a relationship mapping table representing a mapping relationship between the first data table and the second data table; In response to a query, an initial result set is retrieved through the full-text search engine.

Images