Star catalog data retrieval method, device, electronic device and storage medium

Abstract

The present invention discloses a star catalog data retrieval method, device, electronic device, and storage medium. The method is applied to a plan generation node in a database cluster and includes: obtaining retrieval request information, which is determined by a client in response to an information input operation, and includes a sky region division protocol identifier and a retrieval range required for retrieval; obtaining metadata information corresponding to each data storage node from a metadata server node; creating a pseudo-spherical index based on the sky region division protocol identifier, the corresponding protocol, and the retrieval range to obtain a first auxiliary table; generating a query plan based on the first auxiliary table and the metadata information, obtaining retrieval results from the corresponding target data storage nodes based on the query plan, and summarizing the star catalog data corresponding to the retrieval results and feeding them back to the client. This method eliminates the need for users to manually perform UDF extensions outside the database cluster to construct pseudo-spherical indexes, thereby improving the convenience of searching for star catalog data.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a star catalog data retrieval method, device, electronic device, and storage medium. Background Art

[0002] In recent years, with the continuous development of astronomy, vast amounts of star catalog data have been continuously acquired. For example, the Large Spherical Radio Telescope can collect up to 38GB of data per second, with the annual amount of new data reaching tens of petabytes. Some radio telescopes even output approximately 300PB of data annually. Faced with this massive data volume, search methods are necessary to retrieve the star catalog data of interest.

[0003] Currently, the database can be expanded based on a pseudo-spherical partitioning algorithm to implement conical retrieval of star catalog data. Specifically, a user-defined function (UDF) extension module is set up outside the database cluster. The UDF extension module uses the pseudo-spherical partitioning algorithm it supports to convert the right ascension and declination within the search range into one-dimensional pixels and mark the corresponding index identifiers. The database cluster then creates a B-tree index based on each of the index identifiers to obtain an auxiliary table. The auxiliary table contains the index identifiers and the right ascension and declination corresponding to each index identifier. The index identifiers in the auxiliary table are used to search the star catalog stored in the database cluster to obtain the search results.

[0004] The above retrieval method, which extends the database cluster through the UDF extension module, supports a limited number of pseudo-spherical partitioning algorithms. Different UDF extension modules may need to be manually set for different pseudo-spherical partitioning algorithms. That is, during retrieval, the user needs to manually construct a pseudo-spherical index based on the pseudo-spherical partitioning algorithm required for the retrieval, making the process of searching star catalog data more cumbersome. Summary of the Invention

[0005] The present invention provides a star catalog data retrieval method, device, electronic device and storage medium to improve the convenience of star catalog data retrieval.

[0006] In a first aspect, an embodiment of the present invention provides a star catalog data retrieval method, which is applied to a plan generation node in a database cluster, wherein the database cluster further includes a metadata server node and multiple data storage nodes. The method includes:

[0007] Acquire retrieval request information, where the retrieval request information is determined by the client in response to an information input operation, and the retrieval request information includes a sky region division protocol identifier and a retrieval range required for retrieval;

[0008] Acquire metadata information corresponding to each of the data storage nodes from the metadata server node;

[0009] Create a first auxiliary table by performing a pseudo-spherical index based on the protocol corresponding to the sky area division protocol identifier and the search range;

[0010] A query plan is generated based on the first auxiliary table and the metadata information, and a retrieval result is obtained from a corresponding target data storage node based on the query plan. The star catalog data corresponding to the retrieval result is summarized and fed back to the client.

[0011] In a second aspect, an embodiment of the present invention provides a star catalog data retrieval device, configured in a plan generation node in a database cluster, the database cluster further comprising a metadata server node and a plurality of data storage nodes, the device comprising:

[0012] A first acquisition module is configured to acquire search request information, wherein the search request information is determined by the client in response to an information input operation, and the search request information includes a sky area division protocol identifier required for the search and a search scope;

[0013] A second acquisition module is used to obtain metadata information corresponding to each of the data storage nodes from the metadata server node;

[0014] An index creation module, configured to create a pseudo-spherical index based on the protocol corresponding to the sky area division protocol identifier and the search range to obtain a first auxiliary table;

[0015] A retrieval module is used to generate a query plan based on the first auxiliary table and the metadata information, obtain retrieval results from the corresponding target data storage node based on the query plan, summarize the star catalog data corresponding to the retrieval results, and feed them back to the client.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device serving as a plan generation node in a database cluster, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0021] The technical solution of the embodiment of the present invention is that when searching, the user only needs to input the required sky area division protocol identifier and search range through the information input operation on the client. The plan generation node can obtain the information determined by the client in response to the information input operation, automatically process it to determine the corresponding search results and feed back to the client. The pseudo-spherical index creation process is built into the kernel of the database cluster, and natively supports distributed extension and multiple sky area division protocols. The user does not need to manually perform UDF extension outside the database cluster to realize the construction of the pseudo-spherical index, which improves the convenience of searching star catalog data.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flow chart of a star catalog data retrieval method provided according to the first embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of inserting star catalog data into a database cluster provided by Embodiment 1 of the present invention;

[0026] Figure 3 This is a flow chart of a star catalog data retrieval method provided according to the second embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of searching star catalog data in a database cluster provided by the second embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of an architecture for implementing data insertion and data retrieval in a database cluster according to a second embodiment of the present invention;

[0029] Figure 6 This is a schematic structural diagram of a star catalog data retrieval device provided according to a third embodiment of the present invention;

[0030] Figure 7 The figure is a schematic diagram of the structure of an electronic device for implementing the star catalog data retrieval method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0033] Example 1

[0034] Figure 1 This is a flowchart of a star catalog data retrieval method according to a first embodiment of the present invention. This embodiment is applicable to searching star catalog data stored in a database cluster. The method can be performed by a star catalog data retrieval device, which can be implemented in software and / or hardware and integrated into an electronic device that serves as a plan generation node in the database cluster. Furthermore, the electronic device includes, but is not limited to, computers and laptops.

[0035] The database cluster in the embodiment of the present invention may include a plan generation node, a metadata server node and multiple data storage nodes. The retrieval of star catalog data is achieved through the interaction between the plan generation node and the client and other nodes in the cluster.

[0036] A star catalog is a table that records various parameters of celestial bodies (such as position, motion, magnitude, and spectral type). Catalog data is the data recorded in the catalog, which can be understood as the quantitative processing results of celestial observation data. A client can be a terminal, such as a computer or laptop, that provides star catalog data retrieval services for users.

[0037] like Figure 1 As shown, the method includes:

[0038] S110: Acquire search request information, where the search request information is determined by the client in response to an information input operation, and the search request information includes a sky area division protocol identifier and a search scope required for the search.

[0039] The client can provide a user-interactive search interface for searching catalog data. The user can enter information in the search interface, and the information entered constitutes the search request. The search request can contain the information required to search for catalog data, and may include the identifier of the required sky region division protocol and the search range.

[0040] The database cluster kernel supports multiple sky partitioning protocols. A sky partitioning protocol can be understood as a protocol for implementing celestial sphere partitioning on a pseudo-sphere, and can be understood as any algorithm used for pseudo-sphere partitioning, such as Hierarchical Triangular Mesh (HTM) and Hierarchical EqualArea isoLatitude Pixelisation (HEALPix). The identifier of the sky partitioning protocol required for retrieval can be the identifier corresponding to the sky partitioning protocol required for retrieval. The sky partitioning protocol required for retrieval can be any of the multiple sky partitioning protocols supported by the database cluster. The identifier can be the name of the sky partitioning protocol, and this is not limited here.

[0041] The search range can be a conical search of catalog data. A specialized search method in astronomy, conical search defines the right ascension (RA), declination (Dec), and search radius r within the sky. The right ascension and declination determine the coordinates of the search center, and the search range is defined as the range whose angular distance from the search center is less than the search radius r.

[0042] For example, when a user needs to search for star catalog data, they can enter the required sky region division protocol identifier, the right ascension and declination corresponding to the search center required to determine the search range, and the search radius required to determine the search range through one or more input boxes provided in the client's search interface, thereby performing an information input operation. In response to the user's information input operation, the client identifies the information triggered by the information input operation as search request information and transmits the search request information to the plan generation node in the database cluster, so that the plan generation node obtains the search request information.

[0043] S120: Obtain metadata information corresponding to each of the data storage nodes from the metadata server node.

[0044] The database cluster includes multiple data storage nodes, where corresponding star catalog data is distributedly stored. For example, when storing the star catalog data in the database cluster, the star catalog data can be partitioned according to corresponding rules, and the data can be automatically distributed to different data storage nodes in the distributed environment based on the partitions. The rules for partitioning the star catalog data when storing the star catalog data can be determined based on actual application needs. For example, the rules can be used to limit the range of index identifiers for data stored in a partition. The index identifier can be an identity document (ID) used for indexing.

[0045] Metadata server nodes can provide metadata services, storing metadata information corresponding to each data storage node. This metadata information can indicate the data stored in the corresponding data storage node, such as the partition to which the data stored in the corresponding data storage node belongs. Different partitions can be represented by different index ranges. An index can be obtained by mapping right ascension and declination in two-dimensional space to one-dimensional space and assigning a corresponding identifier to the corresponding pixel. The partition to which the data stored in any data storage node belongs can be pre-determined based on application needs and is not limited here.

[0046] In this step, a request is made to the metadata server node to read the metadata information corresponding to each data storage node stored there. The metadata server node then returns the requested information. By obtaining the metadata information corresponding to each data storage node, the partition corresponding to the data stored by each data storage node can be determined, that is, the range of the index identifier corresponding to the data stored by each data storage node can be determined.

[0047] S130: Create a pseudo-spherical index based on the protocol corresponding to the sky area division protocol identifier and the search range to obtain a first auxiliary table.

[0048] In this step, based on the corresponding protocol of the sky area division protocol identifier, the right ascension and declination corresponding to each point in the search range can be converted into one-dimensional pixels and labeled with corresponding index identifiers. Each index identifier and the right ascension and declination corresponding to each index identifier are stored in a first auxiliary table, so that the first auxiliary table includes the index identifier corresponding to each point in the search range. The first auxiliary table can be a table created by the plan generation node to assist in star catalog data retrieval.

[0049] S140 , generating a query plan based on the first auxiliary table and the metadata information, obtaining retrieval results from corresponding target data storage nodes based on the query plan, and summarizing the star catalog data corresponding to the retrieval results and feeding them back to the client.

[0050] A query plan may be a plan for retrieving catalog data in a database cluster. Because catalog data is distributed across multiple data storage nodes in a database cluster, the catalog data involved in the current search may be distributed across one or more data storage nodes. Therefore, each data storage node involved in the current search may be identified as a target data storage node, and a corresponding query sub-plan may be generated for each target data storage node. A query sub-plan may be a plan for retrieving catalog data in a corresponding target data storage node. A query plan may include one or more query sub-plans, which are not limited herein.

[0051] Specifically, the first auxiliary table includes index identifiers corresponding to each point involved in the search range. By traversing each index identifier in the first auxiliary table and combining it with the metadata information corresponding to each data storage node, that is, the partition corresponding to the data stored by each data storage node (that is, the range of the index identifier), the data storage node corresponding to the partition to which each index identifier in the first auxiliary table belongs, that is, the target data storage node, can be determined. Accordingly, the index identifier in the first auxiliary table to be searched for each target data storage node can be determined, and a corresponding query sub-plan can be generated based on the index identifier in the first auxiliary table to be searched for each target data storage node.

[0052] One or more query sub-plans are transmitted to the corresponding target data storage node, so that the target data storage node executes the corresponding query sub-plan, and the star catalog data corresponding to the index identifier to be retrieved indicated by the query sub-plan is retrieved from the star catalog data stored in the target data storage node, and the retrieved star catalog data is fed back to the plan generation node as a retrieval result.

[0053] The plan generation node can aggregate the star catalog data corresponding to the search results fed back by each target data storage node and feed the aggregated star catalog data back to the client. Optionally, after obtaining the aggregated star catalog data fed back by the plan generation node, the client can display the acquired star catalog data on the search interface used by the client to interact with the user, allowing the user to obtain the currently retrieved star catalog data.

[0054] The technical solution of the embodiment of the present invention is that when searching, the user only needs to input the required sky area division protocol identifier and search range through the information input operation on the client. The plan generation node can obtain the information determined by the client in response to the information input operation, automatically process it to determine the corresponding search results and feed back to the client. The pseudo-spherical index creation process is built into the kernel of the database cluster, and natively supports distributed extension and multiple sky area division protocols. The user does not need to manually perform UDF extension outside the database cluster to realize the construction of the pseudo-spherical index, which improves the convenience of searching star catalog data.

[0055] In one embodiment, the method further comprises:

[0056] Obtaining insertion request information, where the insertion request information is determined by the client in response to a data insertion operation, and the insertion request information includes the insertion data and a sky area division protocol identifier required for the insertion;

[0057] Based on the corresponding protocol of the sky area division protocol identifier required for the insertion, the insertion data is automatically partitioned, and the partitioned insertion data is stored in the corresponding data storage node.

[0058] Figure 2 This is a schematic diagram of inserting star catalog data into a database cluster according to the first embodiment of the present invention, combined with Figure 2 The process of inserting catalog data into the database cluster is described as follows:

[0059] In the client, an insertion interface for user interaction can be provided for inserting star catalog data. The data insertion operation can be an operation in which the user enters insertion request information in the insertion interface. The insertion request information can be the information required to insert the star catalog data into the database cluster, and can include the insertion data and the identifier of the sky partitioning protocol required for insertion. The insertion data can be the star catalog data to be inserted into the database cluster. The identifier of the sky partitioning protocol required for insertion can be an identifier corresponding to the sky partitioning protocol required for insertion, such as an identifier corresponding to any of the multiple sky partitioning protocols supported by the database cluster, and is not limited here.

[0060] For example, when a user needs to insert catalog data, they can enter the data to be inserted and the identifier of the desired sky region division protocol through one or more input boxes provided in the client's insertion interface, thereby executing the data insertion operation. In response to the user's data insertion operation, the client identifies the information triggered by the data insertion operation as insertion request information and transmits the insertion request information to the plan generation node in the database cluster, which then obtains the insertion request information.

[0061] Furthermore, the plan generation node can convert the right ascension and declination corresponding to the inserted data into one-dimensional pixels and label them with corresponding index identifiers based on the protocol corresponding to the inserted sky area partitioning protocol. Using the multiple index identifiers corresponding to the inserted data, the inserted data can be divided into multiple partitions according to corresponding rules and automatically assigned to different data storage nodes in the distributed environment. The rules for dividing the inserted data into multiple partitions can be determined based on actual application needs, such as limiting the range of index identifiers for data stored in a particular partition.

[0062] In one embodiment, the data storage node where the partitioned inserted data is stored is determined based on metadata information corresponding to each data storage node.

[0063] The metadata server node can store metadata information corresponding to each data storage node. This metadata information determines which partition the data stored in the corresponding data storage node belongs to. Therefore, the metadata information corresponding to each data storage node can be used to determine which data storage node to store the inserted data after partitioning.

[0064] It should be noted that the present invention does not limit the order of the information input operation corresponding to the retrieval and the data insertion operation corresponding to the insertion of data in the client.

[0065] Optionally, when initially storing data in a database cluster according to embodiments of the present invention, metadata information corresponding to each data storage node can be set based on actual application needs, and this metadata information can be stored in a metadata server node for subsequent use. Accordingly, the metadata information corresponding to each data storage node stored in the metadata server node can also be updated during subsequent use based on actual needs, which is not limited here.

[0066] The embodiments of the present invention can realize automatic partitioning and distributed storage of star catalog data, avoid data distribution skew caused by manual partitioning and tabulation by users, make full use of distributed storage and computing resources, and improve the overall performance and scalability of the system.

[0067] Example 2

[0068] Figure 3 This is a flowchart of a star catalog data retrieval method provided according to the second embodiment of the present invention. This embodiment is based on the above-mentioned first embodiment, and further refines the pseudo-spherical index creation based on the corresponding protocol of the sky area division protocol identifier and the search range to obtain the first auxiliary table; and further refines the generation of a query plan based on the first auxiliary table and the metadata information, and the acquisition of the retrieval results from the corresponding target data storage node based on the query plan. Figure 3 As shown, the method includes the following S301-S309.

[0069] Figure 4 This is a schematic diagram of searching star catalog data in a database cluster according to the second embodiment of the present invention, combined with Figure 3 and Figure 4 The process of retrieving star catalog data from the database cluster is described as follows:

[0070] S301: Acquire search request information. The search request information is determined by the client in response to an information input operation. The search request information includes a sky area division protocol identifier and a search scope required for the search.

[0071] S302: Obtain metadata information corresponding to each of the data storage nodes from the metadata server node.

[0072] S303: Based on the protocol corresponding to the sky area division protocol identifier, convert the right ascension and declination within the search range into one-dimensional pixels and mark corresponding index identifiers.

[0073] In this step, based on the corresponding protocol of the sky area division protocol identifier, the right ascension and declination corresponding to each point involved in the search range can be converted into one-dimensional pixels and marked with corresponding index identifiers.

[0074] S304: Create a B-tree index based on each of the index identifiers to obtain a first auxiliary table, where the first auxiliary table includes each of the index identifiers, the right ascension corresponding to each of the index identifiers, and the declination corresponding to each of the index identifiers.

[0075] In this step, a B-tree index can be created based on each index identifier, a mapping relationship between each index identifier and the right ascension and declination corresponding to each index identifier is established, and each index identifier and the right ascension and declination corresponding to each index identifier are saved in the form of a first auxiliary table.

[0076] S305 , traverse each index identifier included in the first auxiliary table, and determine the target data storage node corresponding to the partition to which each index identifier belongs based on the partition of the data stored in the corresponding data storage node indicated by the metadata information.

[0077] The metadata information corresponding to any data storage node can indicate the partition corresponding to the data stored by that data storage node, that is, the range of corresponding index identifiers. Therefore, by combining any index identifier included in the first auxiliary table with the range of index identifiers indicated by the metadata information corresponding to each data storage node, it is possible to determine which data storage node's partition the index identifier belongs to, and thus determine the target data storage node corresponding to the index identifier.

[0078] S306 : Generate a corresponding query sub-plan for each target data storage node, where the query sub-plan includes an index identifier to be retrieved in the corresponding target data storage node.

[0079] Based on S305, by integrating the target data storage nodes corresponding to the index identifiers included in the first auxiliary table, the number of target data storage nodes and the index identifiers in the first auxiliary table that each target data storage node needs to retrieve can be determined, and then the corresponding query sub-plan can be generated based on the index identifiers in the first auxiliary table that each target data storage node needs to retrieve.

[0080] The query sub-plan generated for a target data storage node can use a temporary auxiliary table to store the index identifier in the first auxiliary table that the target data storage node needs to search. Optionally, the temporary auxiliary table can also store the right ascension and declination corresponding to the index identifier that the target data storage node needs to search.

[0081] S307 , transmitting the query sub-plan to a corresponding target data storage node, so that the target data storage node executes the query sub-plan in combination with the stored star catalog data, determines a retrieval result, and feeds the retrieval result back to the plan generation node.

[0082] When storing star catalog data in a data storage node in a database cluster, the right ascension and declination in the stored star catalog data can be converted into one-dimensional pixels and labeled with corresponding index identifiers. A B-tree index is then created based on each index identifier to obtain a second auxiliary table. The second auxiliary table includes the index identifiers corresponding to the star catalog data stored in the data storage node, and the right ascension and declination corresponding to each index identifier. The second auxiliary table can be a table created by the data storage node to assist in star catalog data retrieval.

[0083] Therefore, for any target data storage node, the index identifier to be retrieved stored in the temporary auxiliary table in the query sub-plan it receives can be used to search for an index identifier consistent with the index identifier to be retrieved stored in the temporary auxiliary table in the second auxiliary table corresponding to the target data storage node, and the star catalog data corresponding to the consistent index identifier can be fed back to the plan generation node as a retrieval result.

[0084] S308: Obtain the search result fed back by the target data storage node.

[0085] S309: Summarize the star catalog data corresponding to the search results and feed them back to the client.

[0086] The technical solution of the embodiments of the present invention enables the database kernel to automatically schedule and merge data during distributed queries. When performing a cone-shaped search of catalog data, query requests are distributed to the appropriate partitioned nodes, and the database kernel automatically schedules and coordinates query operations on each node. Query results are computed and retrieved in parallel on each node, then merged and integrated by the database kernel and ultimately returned to the application layer. This distributed query process ensures query efficiency while reducing application layer complexity, allowing users to transparently utilize the database system without having to worry about the underlying distributed storage and computing details.

[0087] In one embodiment, after transmitting the query sub-plan to the corresponding target data storage node, the method further includes:

[0088] Obtaining indication information transmitted by the target data storage node indicating whether the retrieval is successful or not;

[0089] Feedback the indication information to the client.

[0090] The indication information may be information indicating whether the search in the target data storage node is successful or not. The form of the indication information is not limited. By obtaining the indication information from the target data storage node and feeding the indication information back to the client, the user can be informed whether the search in the target data storage node is successful or not.

[0091] If the instruction information indicates that the retrieval of the target data storage node is successfully executed, no further processing is required; if the instruction information indicates that the retrieval of the target data storage node is not successfully executed, the user can initiate the retrieval again through the client.

[0092] Figure 5 This is a schematic diagram of an architecture for implementing data insertion and data retrieval in a database cluster according to the second embodiment of the present invention. Figure 5 As shown in the figure, in this architecture, the sky area division protocol can be combined with the distributed database to achieve efficient storage and query of large-scale complex star catalog data sets; the pseudo-spherical index creation process can be built into the database kernel, and the database can be automatically partitioned and tabulated through index identifiers, which is transparent to the application layer; the database kernel has the ability of automatic scheduling and merging, making the distributed query process more efficient and simplified.

[0093] Example 3

[0094] Figure 6 This is a schematic diagram of the structure of a star catalog data retrieval device provided according to the third embodiment of the present invention. This embodiment is applicable to the case of retrieving star catalog data stored in a database cluster. The device is configured in a plan generation node in the database cluster, and the database cluster also includes a metadata server node and multiple data storage nodes. Figure 6 As shown, the specific structure of the device includes:

[0095] A first acquisition module 61 is configured to acquire search request information, wherein the search request information is determined by the client in response to an information input operation, and the search request information includes a sky area division protocol identifier and a search range required for the search;

[0096] A second acquisition module 62 is configured to acquire metadata information corresponding to each of the data storage nodes from the metadata server node;

[0097] An index creation module 63 is configured to create a pseudo-spherical index based on the protocol corresponding to the sky area division protocol identifier and the search range to obtain a first auxiliary table;

[0098] The retrieval module 64 is used to generate a query plan based on the first auxiliary table and the metadata information, obtain retrieval results from the corresponding target data storage node based on the query plan, summarize the star catalog data corresponding to the retrieval results, and feed them back to the client.

[0099] The star catalog data retrieval device provided in this embodiment uses a first acquisition module to acquire retrieval request information, which is determined by a client in response to an information input operation and includes a desired sky partition protocol identifier and a search scope. A second acquisition module acquires metadata information corresponding to each data storage node from the metadata server node. An index creation module creates a pseudo-spherical index based on the protocol corresponding to the sky partition protocol identifier and the search scope to obtain a first auxiliary table. A retrieval module generates a query plan based on the first auxiliary table and the metadata information, and retrieves retrieval results from the corresponding target data storage nodes based on the query plan. The star catalog data corresponding to the retrieval results is aggregated and fed back to the client. This solution requires only that the user input the desired sky partition protocol identifier and search scope through information input on the client. The plan generation node then acquires the information determined by the client in response to the information input operation, automatically processes it to determine the corresponding retrieval results, and feeds them back to the client. The pseudo-spherical index creation process is built into the database cluster kernel, natively supporting distributed extensions and multiple sky partition protocols. This eliminates the need for the user to manually construct a pseudo-spherical index through external UDF extensions to the database cluster, thereby improving the convenience of star catalog data retrieval.

[0100] Furthermore, the index creation module 63 is specifically configured to:

[0101] Based on the corresponding protocol of the sky area division protocol identifier, convert the right ascension and declination within the search range into one-dimensional pixels and mark the corresponding index identifiers;

[0102] A B-tree index is created based on each of the index identifiers to obtain a first auxiliary table, where the first auxiliary table includes each of the index identifiers, the right ascension corresponding to each of the index identifiers, and the declination corresponding to each of the index identifiers.

[0103] Furthermore, the query plan includes one or more query sub-plans. Accordingly, the retrieval module 64 is specifically configured to:

[0104] Traversing each index identifier included in the first auxiliary table, and determining a target data storage node corresponding to the partition to which each index identifier belongs based on the partition of the data stored in the corresponding data storage node indicated by the metadata information;

[0105] A corresponding query sub-plan is generated for each target data storage node, where the query sub-plan includes an index identifier to be retrieved in the corresponding target data storage node.

[0106] Furthermore, the retrieval module 64 is specifically configured to:

[0107] Transmitting the query sub-plan to a corresponding target data storage node, so that the target data storage node executes the query sub-plan in combination with the stored star catalog data, determines a retrieval result, and feeds the retrieval result back to the plan generation node;

[0108] Obtain the search result fed back by the target data storage node.

[0109] Furthermore, the device also includes an indication information feedback module, which is used to:

[0110] After transmitting the query sub-plan to the corresponding target data storage node, obtaining indication information transmitted by the target data storage node indicating whether the retrieval is successful or not; and feeding back the indication information to the client.

[0111] Furthermore, the device also includes an insertion module for:

[0112] Obtaining insertion request information, where the insertion request information is determined by the client in response to a data insertion operation, and the insertion request information includes the insertion data and a sky area division protocol identifier required for the insertion;

[0113] Based on the corresponding protocol of the sky area division protocol identifier required for the insertion, the insertion data is automatically partitioned, and the partitioned insertion data is stored in the corresponding data storage node.

[0114] Furthermore, based on the metadata information corresponding to each data storage node, the data storage node where the partitioned inserted data is stored is determined.

[0115] The star catalog data retrieval device provided in the embodiment of the present invention can execute the star catalog data retrieval method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Example 4

[0117] Figure 7Schematic diagram of an electronic device for implementing the star catalog data retrieval method according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided for example purposes only and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0118] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0119] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0120] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the star catalog data retrieval method.

[0121] In some embodiments, the star catalog data retrieval method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the star catalog data retrieval method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the star catalog data retrieval method in any other suitable manner (e.g., via firmware).

[0122] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0128] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0129] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A star catalog data retrieval method, characterized in that: Applied to a plan generation node in a database cluster, the database cluster further comprising a metadata server node and a plurality of data storage nodes, the method comprising: Obtaining retrieval request information, the retrieval request information being determined by the client in response to an information input operation, the retrieval request information including a sky region partitioning protocol identifier required for retrieval and a retrieval scope; wherein the sky region partitioning protocol identifier corresponds to any one of a plurality of sky region partitioning protocols supported by the kernel of the database cluster, and the sky region partitioning protocol is any algorithm for pseudo-spherical partitioning; Obtaining metadata information corresponding to each of the data storage nodes from the metadata server node; wherein the metadata information indicates a partition of data stored in the corresponding data storage node, and ranges of different index identifiers represent different partitions; Create a first auxiliary table by performing pseudo-spherical indexing based on the protocol corresponding to the sky area division protocol identifier and the search range; Generate a query plan based on the first auxiliary table and the metadata information, obtain retrieval results from the corresponding target data storage node based on the query plan, summarize the star catalog data corresponding to the retrieval results, and feed them back to the client; The query plan includes one or more query sub-plans. Accordingly, generating a query plan based on the first auxiliary table and the metadata information includes: Traversing each index identifier included in the first auxiliary table, and determining a target data storage node corresponding to the partition to which each index identifier belongs based on the partition of the data stored in the corresponding data storage node indicated by the metadata information; A corresponding query sub-plan is generated for each target data storage node, where the query sub-plan includes an index identifier to be retrieved in the corresponding target data storage node.

2. The method according to claim 1, characterized in that A pseudo-spherical index is created based on the protocol corresponding to the sky area division protocol identifier and the search range to obtain a first auxiliary table, including: Based on the corresponding protocol of the sky area division protocol identifier, convert the right ascension and declination within the search range into one-dimensional pixels and mark the corresponding index identifiers; A B-tree index is created based on each of the index identifiers to obtain a first auxiliary table, where the first auxiliary table includes each of the index identifiers, the right ascension corresponding to each of the index identifiers, and the declination corresponding to each of the index identifiers.

3. The method according to claim 1, characterized in that Acquiring retrieval results from the corresponding target data storage node based on the query plan includes: Transmitting the query sub-plan to a corresponding target data storage node, so that the target data storage node executes the query sub-plan in combination with the stored star catalog data, determines a retrieval result, and feeds the retrieval result back to the plan generation node; Obtain the search result fed back by the target data storage node.

4. The method according to claim 3, characterized in that After transmitting the query sub-plan to the corresponding target data storage node, the method further includes: Obtaining indication information transmitted by the target data storage node indicating whether the retrieval is successful or not; Feedback the indication information to the client.

5. The method according to claim 1, characterized in that Also includes: Obtaining insertion request information, where the insertion request information is determined by the client in response to a data insertion operation, and the insertion request information includes the insertion data and a sky area division protocol identifier required for the insertion; Based on the corresponding protocol of the sky area division protocol identifier required for the insertion, the insertion data is automatically partitioned, and the partitioned insertion data is stored in the corresponding data storage node.

6. The method according to claim 5, characterized in that Based on the metadata information corresponding to each data storage node, the data storage node where the partitioned inserted data is stored is determined.

7. A star catalog data retrieval device, characterized in that: A plan generation node is configured in a database cluster, wherein the database cluster further includes a metadata server node and a plurality of data storage nodes, and the device includes: A first acquisition module is configured to acquire search request information, the search request information being determined by the client in response to an information input operation, the search request information including a sky region partitioning protocol identifier required for the search and a search scope; wherein the sky region partitioning protocol identifier corresponds to any one of a plurality of sky region partitioning protocols supported by the kernel of the database cluster, and the sky region partitioning protocol is any algorithm for pseudo-spherical partitioning; A second acquisition module is configured to acquire metadata information corresponding to each of the data storage nodes from the metadata server node; wherein the metadata information indicates a partition of data stored in the corresponding data storage node, and ranges of different index identifiers represent different partitions; An index creation module, configured to create a pseudo-spherical index based on the protocol corresponding to the sky area division protocol identifier and the search range to obtain a first auxiliary table; a retrieval module, configured to generate a query plan based on the first auxiliary table and the metadata information, obtain retrieval results from the corresponding target data storage node based on the query plan, summarize the star catalog data corresponding to the retrieval results, and feed them back to the client; The query plan includes one or more query sub-plans. Accordingly, the retrieval module is specifically configured to: Traversing each index identifier included in the first auxiliary table, and determining a target data storage node corresponding to the partition to which each index identifier belongs based on the partition of the data stored in the corresponding data storage node indicated by the metadata information; A corresponding query sub-plan is generated for each target data storage node, where the query sub-plan includes an index identifier to be retrieved in the corresponding target data storage node.

8. An electronic device, characterized in that: As a plan generation node in the database cluster, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images