Data query method, device and equipment of distributed file system and medium

Abstract

This invention provides a data query method, apparatus, device, and medium for a distributed file system. The method includes: acquiring data query information sent by a user terminal; performing semantic parsing on the data query information to generate corresponding file query information; determining a matching target file from the distributed file system based on the file query information; and outputting the target file to the user terminal. The method of this invention first acquires the data query information sent by the user terminal and performs semantic parsing on the data query information to determine the file query information to be queried. Then, it determines a matching target file from the distributed file system based on the file query information and outputs the target file to the user terminal. This method eliminates the need for manual retrieval of corresponding data based on data storage paths, automating data querying of the distributed file system and improving data query efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data query method, apparatus, device, and medium for a distributed file system. Background Technology

[0002] Alluxio is a memory-centric virtual distributed file system. It unifies data access methods, bridging upper-layer computing frameworks and underlying storage systems. Applications only need to connect to Alluxio to access data stored on any underlying storage system. Furthermore, Alluxio's memory-centric architecture enables data access speeds several orders of magnitude faster than conventional solutions.

[0003] Alluxio's underlying data is stored in a columnar schema. When querying Alluxio data, upper-layer computing frameworks such as the Spark engine can initially only obtain the data storage path, not the corresponding data directly. At this point, manual retrieval of the data based on the storage path is required, resulting in low data query efficiency. Summary of the Invention

[0004] This invention provides a data query method, apparatus, device, and medium for a distributed file system, which solves the problem that the current method requires manual retrieval of data based on the data storage path, resulting in low data query efficiency.

[0005] The first aspect of this invention provides a data query method for a distributed file system, comprising:

[0006] Obtain data query information sent by the user terminal;

[0007] The data query information is semantically parsed to generate corresponding file query information;

[0008] The matching target file is determined from the distributed file system based on the file query information.

[0009] Output the target file to the user terminal.

[0010] Furthermore, in the method described above, the data format of the data query information is Structured Query Language (SQL);

[0011] The step of semantically parsing the data query information to generate corresponding file query information includes:

[0012] The data query information is parsed using SQL to identify its semantics;

[0013] The target data to be queried is determined based on the semantics of the data query information.

[0014] The corresponding file query information is generated based on the target data; the data format of the file query information is matched with the distributed file system.

[0015] Further, in the method described above, determining the matching target file from the distributed file system based on the file query information includes:

[0016] The storage location of the target file is determined from the distributed file system based on the file query information.

[0017] The target file is determined based on the storage location.

[0018] Further, in the method described above, determining the matching target file based on the storage location includes:

[0019] A corresponding temporary table is generated based on the storage location; the temporary table is used to establish a connection between the storage location and the target file in the distributed file system.

[0020] The target file is queried from the distributed file system based on the temporary table.

[0021] Further, in the method described above, outputting the target file to the user terminal includes:

[0022] The target file is output to the user terminal using the Spark computing engine.

[0023] Furthermore, in the method described above, after determining the matching target file from the distributed file system based on the file query information, the method further includes:

[0024] Matching data is determined from the target file based on the file query information; the target file includes data related to a preset storage data category; the preset storage data category matches the target file.

[0025] A new file is generated based on the matched data;

[0026] The step of outputting the target file to the user terminal includes:

[0027] The new file is output to the user terminal.

[0028] Furthermore, in the method described above, after determining the matching target file from the distributed file system based on the file query information, the method further includes:

[0029] Matching data is determined from the target file based on the file query information; the target file includes data related to a preset storage data category; the preset storage data category matches the target file.

[0030] The matched data is stored in a read / write database that matches the distributed file system;

[0031] The step of outputting the target file to the user terminal includes:

[0032] Retrieve the matching data from the read / write database;

[0033] The matched data is output to the user terminal.

[0034] A second aspect of the present invention provides a data query apparatus for a distributed file system, comprising:

[0035] The acquisition module is used to acquire data query information sent by the user terminal;

[0036] The generation module is used to perform semantic parsing on the data query information to generate corresponding file query information;

[0037] The determination module is used to determine the matching target file from the distributed file system based on the file query information;

[0038] The output module is used to output the target file to the user terminal.

[0039] Furthermore, in the apparatus described above, the data format of the data query information is Structured Query Language (SQL);

[0040] The generation module is specifically used for:

[0041] The data query information is parsed using SQL to identify its semantics; the target data to be queried is determined based on the semantics of the data query information; corresponding file query information is generated based on the target data; and the data format of the file query information is matched with the distributed file system.

[0042] Furthermore, in the apparatus described above, the determining module is specifically used for:

[0043] The storage location of the target file is determined from the distributed file system based on the file query information; the matching target file is then determined based on the storage location.

[0044] Furthermore, in the apparatus described above, when determining the matching target file based on the storage location, the determining module is specifically used for:

[0045] A corresponding temporary table is generated based on the storage location; the temporary table is used to establish a connection between the storage location and the target file in the distributed file system; the target file is queried from the distributed file system based on the temporary table.

[0046] Furthermore, in the apparatus described above, the output module is specifically used for:

[0047] The target file is output to the user terminal using the Spark computing engine.

[0048] Furthermore, the apparatus as described above further includes:

[0049] The first data writing module is used to determine matching data from the target file based on the file query information; the target file includes data related to a preset storage data category; the preset storage data category matches the target file; and a new file is generated based on the matching data.

[0050] The output module is also used for:

[0051] The new file is output to the user terminal.

[0052] Furthermore, the apparatus as described above further includes:

[0053] The second data writing module is used to determine matching data from the target file based on the file query information; the target file includes data related to a preset storage data category; the preset storage data category matches the target file; and the matching data is stored in a read / write database that matches the distributed file system.

[0054] The output module is also used for:

[0055] The matching data is retrieved from the read / write database and output to the user terminal.

[0056] A third aspect of the present invention provides an electronic device, comprising: a memory and a processor;

[0057] Memory; memory for storing executable instructions of the processor;

[0058] The processor is configured to execute the data query method of the distributed file system as described in any of the first aspects.

[0059] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the data query method of the distributed file system according to any one of the first aspects.

[0060] The fifth aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the data query method of the distributed file system as described in any of the first aspects.

[0061] This invention provides a data query method, apparatus, device, and medium for a distributed file system. The method includes: acquiring data query information sent by a user terminal; performing semantic parsing on the data query information to generate corresponding file query information; determining a matching target file from the distributed file system based on the file query information; and outputting the target file to the user terminal. The distributed file system data query method of this invention first acquires data query information sent by a user terminal and performs semantic parsing on the data query information to determine the file query information to be queried. Then, it determines a matching target file from the distributed file system based on the file query information and outputs the target file to the user terminal. This distributed file system data query method of this invention eliminates the need for manual retrieval of corresponding data based on data storage paths, automating data querying of the distributed file system and improving data query efficiency. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0063] Figure 1 This is a scenario diagram illustrating the data query method of the distributed file system in this embodiment of the invention.

[0064] Figure 2 A flowchart illustrating the data query method of the distributed file system provided in the first embodiment of the present invention;

[0065] Figure 3 A flowchart illustrating the data query method for a distributed file system provided in the second embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the overall architecture of the data query method for a distributed file system provided in the second embodiment of the present invention;

[0067] Figure 5 This is a schematic diagram of the structure of the data query device for the distributed file system provided in the third embodiment of the present invention;

[0068] Figure 6 This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of the present invention.

[0069] The accompanying drawings illustrate specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0071] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0072] To clearly understand the technical solution of this application, a detailed description of existing technical solutions is first provided. In the big data ecosystem, Alluxio is a distributed file system that sits between computing frameworks (such as the Spark computing engine and the MapReduce computing engine) and existing storage systems (HDFS (Hadoop Distributed File System) database). Alluxio brings significant performance improvements to the big data software stack. Users can run Alluxio in an independent cluster. However, when querying Alluxio data, it is first necessary to understand the data distribution directory and manually register temporary tables according to the data storage path before data querying can be performed, resulting in low data query efficiency.

[0073] Therefore, in response to the problem that existing technologies require manual retrieval of corresponding data based on the data storage path, resulting in low data query efficiency, the inventors discovered in their research that an automated data query method can be constructed to solve this problem. By performing semantic parsing on the data query information sent by the user terminal, file queries on the distributed file system can be automatically realized.

[0074] Specifically, the method first acquires the data query information sent by the user terminal and performs semantic parsing on the data query information to determine the file query information to be queried. Then, based on the file query information, it determines the matching target file from the distributed file system and outputs the target file to the user terminal. This embodiment's distributed file system data query method eliminates the need for manual retrieval of corresponding data based on the data storage path, automating the data query process and improving data query efficiency.

[0075] Based on the above-mentioned inventive discovery, the inventor has proposed the technical solution of this application.

[0076] The following describes the application scenarios of the data query method for the distributed file system provided in this embodiment of the invention. For example... Figure 1 As shown, 1 represents an electronic device, 2 represents a user terminal, and 3 represents a distributed file system. User terminal 2 can be an electronic device; however, the diagram only uses mobile terminals and does not limit the type of user terminal. Distributed file system 3 is exemplarily described as a cluster of multiple electronic devices. The network architecture of the application scenario corresponding to the data query method for the distributed file system provided in this embodiment includes: electronic device 1, user terminal 2, and distributed file system 3. Distributed file system 3 stores various files.

[0077] When a data query is required, user terminal 2 sends a data query message to electronic device 1. The data query message may include the query statement entered by the user. Electronic device 1 performs semantic parsing on the data query message to generate corresponding file query information. This file query information more clearly specifies the file to be queried compared to the data query message. At the same time, electronic device 1 determines the matching target file from the distributed file system 3 based on the file query information and outputs the target file to user terminal 2 to complete the data query process.

[0078] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0079] Figure 2 This is a flowchart illustrating the data query method for a distributed file system provided in the first embodiment of the present invention, as shown below. Figure 2 As shown, in this embodiment, the execution entity of this invention is a data query device for a distributed file system, which can be integrated into an electronic device. The data query method for a distributed file system provided in this embodiment includes the following steps:

[0080] Step S101: Obtain the data query information sent by the user terminal.

[0081] In this embodiment, the data query information includes the query statement entered by the user on the user terminal, such as searching for communication data of a certain area on a certain day, searching for a certain movie, etc. The user terminal can be an electronic device such as a computer, tablet computer, etc., or a mobile terminal such as a mobile phone.

[0082] Step S102: Perform semantic parsing on the data query information to generate corresponding file query information.

[0083] In this embodiment, semantic parsing of the data query information mainly involves determining the specific data content to be searched from the data query information. For example, if the query information contains the query statement for a certain brand of vehicle, then the data content to be searched can be determined to include data such as the appearance, configuration information, and performance parameters of that brand of vehicle. The data content to be searched obtained after semantic parsing is then used to generate corresponding file query information.

[0084] Step S103: Determine the matching target file from the distributed file system based on the file query information.

[0085] In this embodiment, since the file query information includes the data content to be searched, a matching file can be determined from the distributed file system. Distributed file systems like Alluxio store all data as files, with each file containing the corresponding data. When the data content to be queried is known, the matching target file can be directly determined using Alluxio.

[0086] Step S104: Output the target file to the user terminal.

[0087] In this embodiment, the determined target file is output to the user terminal so that the user terminal can perform subsequent processing based on the target file, such as further data search or data analysis.

[0088] This invention provides a data query method for a distributed file system. The method includes: acquiring data query information sent by a user terminal; performing semantic parsing on the data query information to generate corresponding file query information; determining a matching target file from the distributed file system based on the file query information; and outputting the target file to the user terminal.

[0089] The data query method for a distributed file system of the present invention first obtains data query information sent by a user terminal and performs semantic parsing on the data query information to determine the file query information to be queried. Then, based on the file query information, it determines the matching target file from the distributed file system and outputs the target file to the user terminal. The data query method for a distributed file system of the present invention eliminates the need for manual retrieval of corresponding data based on the data storage path, performing data queries on the distributed file system in an automated manner, thus improving data query efficiency.

[0090] Figure 3 This is a flowchart illustrating the data query method for a distributed file system provided in the second embodiment of the present invention, as shown below. Figure 3 As shown, the data query method for a distributed file system provided in this embodiment is based on the data query method for a distributed file system provided in the previous embodiment of the present invention, with further refinement of each step. Therefore, the data query method for a distributed file system provided in this embodiment includes the following steps.

[0091] In this embodiment, the data format for the data query information is Structured Query Language (SQL).

[0092] Step S201: Obtain the data query information sent by the user terminal.

[0093] In this embodiment, the implementation of step 201 is similar to that of step 101 in the previous embodiment of the present invention, and will not be described in detail here.

[0094] Step S202: Perform SQL parsing on the data query information to identify the semantics of the data query information.

[0095] In this embodiment, the personnel who perform data queries generally use SQL statements, i.e., Structured Query Language. Since distributed file systems such as Alluxio cannot directly recognize and parse SQL statements, it is necessary to perform SQL parsing on the data query information in order to recognize the semantics of the data query information.

[0096] Step S203: Determine the target data to be queried based on the semantics of the data query information.

[0097] In this embodiment, the target data to be searched can be determined from the data query information based on the semantics of the data query information. For example, if the query statement in the data query information is a certain mobile phone, then the target data to be searched can be determined to include data such as the appearance of the mobile phone, the functions of the mobile phone, and the performance parameters.

[0098] Step S204: Generate corresponding file query information based on the target data. The data format of the file query information matches the distributed file system.

[0099] In this embodiment, if the target data to be queried includes information such as the phone's appearance, functions, and performance parameters, corresponding file query information can be directly generated based on this target data. This file query information needs to be converted to a format compatible with the distributed file system. For example, the Alluxio distributed file system cannot parse or recognize SQL format; therefore, after parsing the data query information using SQL, the generated file query information cannot be in SQL format and requires format conversion.

[0100] Step S205: Determine the storage location of the target file from the distributed file system based on the file query information.

[0101] In this embodiment, since the distributed file system can manage all stored files and know the storage location of all files, the storage location of the target file can be determined through the distributed file system based on the file query information.

[0102] Step S206: Determine the matching target file based on the storage location.

[0103] In this embodiment, once the storage location is determined, the target file to be queried can be determined from the distributed file system. In the prior art, this step is generally performed by the computing framework through the distributed file system, querying layer by layer to finally obtain the target file. However, in this embodiment, the computing framework is no longer required to perform this step, and the matching target file can be determined directly based on the storage location.

[0104] Optionally, in this embodiment, determining the matching target file based on the storage location can specifically be as follows:

[0105] A temporary table is generated based on the storage location. This temporary table is used to establish a connection between the storage location and the target file in the distributed file system.

[0106] The target file is retrieved from the distributed file system based on the temporary table.

[0107] In this embodiment, by constructing a temporary table corresponding to the storage location, the target file can be directly queried from the distributed file system based on the temporary table, eliminating the need for layer-by-layer queries. Furthermore, in existing technologies, after a distributed file system such as Alluxio reports the file storage location, a corresponding temporary table needs to be manually generated based on the storage location, and then the target file is queried from the distributed file system using the temporary table. This embodiment improves data query efficiency through automation and also reduces the query threshold and difficulty for users.

[0108] In this embodiment, the temporary table can be automatically destroyed after the data query is completed, so as to avoid additional occupation of database capacity.

[0109] Step S207: Use the Spark computing engine to output the target file to the user terminal.

[0110] In this embodiment, the framework of the distributed file system is as follows: Figure 4 As shown, the top layer is the computing framework, typically using the Spark computing engine. Spark has persistent processes, allowing users to make multiple query requests with a single connection, reducing the submission time of a single Spark task. The middle layer is the distributed file system; this embodiment uses Alluxio. The bottom layer consists of various databases, such as HDFS and MySQL. MySQL databases are generally not compatible with Alluxio. This embodiment provides a DBLink-like cross-platform database interaction function, pre-building connections between local databases like HDFS and other platform databases like MySQL, enabling Alluxio to load data from other non-local databases.

[0111] When a user terminal sends a data query, the top-level computing framework receives the query and sends a query request to Alluxio. At this point, the distributed file system's data query device in this embodiment receives the query and executes the subsequent query process. Once the target file is identified in Alluxio, the Spark computing engine can output the target file to the user terminal.

[0112] Normally, Spark's computing engine can only export the entire dataset, meaning it can only export the target file and not just a portion of it. To address this, this embodiment offers a further improvement, implemented as follows:

[0113] The first method involves identifying the target file after determining its matching status, and then retrieving matching data from the target file based on file query information. The target file includes data related to a preset storage data category that matches the target file.

[0114] A new file is generated based on the matched data.

[0115] Output the new file to the user terminal.

[0116] In this embodiment, the data that needs to be output from the target file is first determined, and the matching data is then recombined to generate a new file. For example, if the target file is file a, and file a has a pre-defined data storage category, then file a contains data a, data b, and data c corresponding to the pre-defined data storage category. If the matching data is data a and data b, then data a and data b can be extracted separately to generate a new file, file b, and then this file b is output to the user terminal. This reduces data channel usage and improves the accuracy of data queries.

[0117] The second method involves identifying the target file and then determining the matching data from it based on file query information. The target file includes data related to a preset storage data category that matches the target file.

[0118] The matched data is stored in a read / write database that matches the distributed file system.

[0119] Retrieve matching data from the read / write database.

[0120] Output the matched data to the user terminal.

[0121] In this embodiment, a read / write database that matches the distributed file system can be pre-established, such as an HDFS database, or a pre-built associated non-local database such as a MySQL database or a GP (Greenplum) database on other platforms, which greatly facilitates cross-platform data loading and export.

[0122] The matching data from the identified target files is then stored in a read / write database, from which matching data is retrieved. This ensures that the final output to the user terminal contains all the data needed for the query, rather than a bloated file.

[0123] Figure 5 This is a schematic diagram of the structure of the data query device for the distributed file system provided in the third embodiment of the present invention, as shown below. Figure 5 As shown, in this embodiment, the data query device 300 of the distributed file system includes:

[0124] The acquisition module 301 is used to acquire data query information sent by the user terminal.

[0125] The generation module 302 is used to perform semantic parsing on the data query information to generate corresponding file query information.

[0126] The determination module 303 is used to determine the matching target file from the distributed file system based on the file query information.

[0127] Output module 304 is used to output the target file to the user terminal.

[0128] The distributed file system data query device provided in this embodiment can execute... Figure 2 The technical solution of the method embodiment shown has the same implementation principle and technical effect as... Figure 2 The methods and embodiments shown are similar and will not be described in detail here.

[0129] Meanwhile, the data query device for the distributed file system provided by the present invention further refines the data query device for the distributed file system based on the data query device for the distributed file system provided in the previous embodiment.

[0130] Optionally, in this embodiment, the data format for the data query information is Structured Query Language (SQL).

[0131] The generation module 302 is specifically used for:

[0132] The data query information is parsed using SQL to identify its semantics. Based on this semantics, the target data to be queried is determined. Corresponding file query information is then generated based on the target data. The data format of the file query information is matched to the distributed file system.

[0133] Optionally, in this embodiment, the determining module 303 is specifically used for:

[0134] The storage location of the target file is determined from the distributed file system based on the file query information. The matching target file is then identified based on the storage location.

[0135] Optionally, in this embodiment, when determining the matching target file based on the storage location, the determining module 303 is specifically used for:

[0136] A temporary table is generated based on the storage location. This temporary table is used to establish a connection between the storage location and the target file in the distributed file system. The target file is then retrieved from the distributed file system based on the temporary table.

[0137] Optionally, in this embodiment, the output module 304 is specifically used for:

[0138] The Spark computing engine is used to output the target file to the user terminal.

[0139] Optionally, in this embodiment, the data query device 300 of the distributed file system further includes:

[0140] The first data writing module is used to determine matching data from the target file based on file query information. The target file includes data related to a preset storage data category, and the preset storage data category matches the target file. A new file is generated based on the matching data.

[0141] Output module 304 is also used for:

[0142] Output the new file to the user terminal.

[0143] Optionally, in this embodiment, the data query device 300 of the distributed file system further includes:

[0144] The second data writing module is used to determine matching data from the target file based on file query information. The target file includes data related to a preset storage data category, and the preset storage data category matches the target file. The matching data is then stored in a read / write database that matches the distributed file system.

[0145] Output module 304 is also used for:

[0146] Retrieve matching data from the database. Output the matching data to the user terminal.

[0147] The distributed file system data query device provided in this embodiment can execute... Figures 2-4 The technical solution of the method embodiment shown has the same implementation principle and technical effect as... Figures 2-4The methods and embodiments shown are similar and will not be described in detail here.

[0148] According to embodiments of the present invention, the present invention also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0149] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of the present invention. The electronic device is intended for various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0150] like Figure 6 As shown, the electronic device includes a processor 401 and a memory 402. The various components are interconnected via different buses and can be mounted on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the electronic device.

[0151] The memory 402 is the non-transitory computer-readable storage medium provided by this invention. The memory stores instructions executable by at least one processor to cause the at least one processor to execute the data query method of the distributed file system provided by this invention. The non-transitory computer-readable storage medium of this invention stores computer instructions for causing a computer to execute the data query method of the distributed file system provided by this invention.

[0152] Memory 402, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the data query method of the distributed file system in this embodiment of the invention (e.g., appendix). Figure 5 The acquisition module 301, generation module 302, determination module 303, and output module 304 are shown. The processor 401 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 402, thereby implementing the data query method of the distributed file system in the above method embodiment.

[0153] Meanwhile, this embodiment also provides a computer product that, when the instructions in the computer product are executed by the processor of an electronic device, enables the electronic device to execute the data query method of the distributed file system described in embodiments one and two above.

[0154] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The invention is intended to cover any variations, uses, or adaptations of the embodiments thereof that follow the general principles of the embodiments and include common knowledge or customary techniques in the art not disclosed in the embodiments thereof. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the embodiments thereof are indicated by the following claims.

[0155] It should be understood that the embodiments of the present invention are not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from their scope. The scope of the embodiments of the present invention is limited only by the appended claims.

Claims

1. A data query method for a distributed file system, characterized in that, include: Obtain data query information sent by the user terminal; The data query information is semantically parsed to generate corresponding file query information; The data format of the file query information is compatible with the distributed file system; The matching target file is determined from the distributed file system based on the file query information. The target file is output to the user terminal using the Spark computing engine; The step of determining the matching target file from the distributed file system based on the file query information includes: The storage location of the target file is determined from the distributed file system based on the file query information. A corresponding temporary table is generated based on the storage location; the temporary table is used to establish a connection between the storage location and the target file in the distributed file system. The target file is queried from the distributed file system based on the temporary table.

2. The method according to claim 1, characterized in that, The data query information is in the format of Structured Query Language (SQL). The step of semantically parsing the data query information to generate corresponding file query information includes: The data query information is parsed using SQL to identify its semantics; The target data to be queried is determined based on the semantics of the data query information. Generate corresponding file query information based on the target data.

3. The method according to any one of claims 1 to 2, characterized in that, After determining the matching target file from the distributed file system based on the file query information, the process further includes: Matching data is determined from the target file based on the file query information; the target file includes data related to a preset storage data category; the preset storage data category matches the target file. A new file is generated based on the matched data; The step of outputting the target file to the user terminal includes: The new file is output to the user terminal.

4. The method according to any one of claims 1 to 2, characterized in that, After determining the matching target file from the distributed file system based on the file query information, the process further includes: Matching data is determined from the target file based on the file query information; the target file includes data related to a preset storage data category; the preset storage data category matches the target file. The matched data is stored in a read / write database that matches the distributed file system; The step of outputting the target file to the user terminal includes: Retrieve the matching data from the read / write database; The matched data is output to the user terminal.

5. A data query device for a distributed file system, characterized in that, include: The acquisition module is used to acquire data query information sent by the user terminal; The generation module is used to perform semantic parsing on the data query information to generate corresponding file query information; The data format of the file query information is compatible with the distributed file system; The determination module is used to determine the matching target file from the distributed file system based on the file query information; The output module is used to output the target file to the user terminal using the Spark computing engine. The determining module is specifically used to determine the storage location of the target file from the distributed file system based on the file query information; and to generate a corresponding temporary table based on the storage location. The temporary table is used to establish a connection between the storage location and the target file in the distributed file system; the target file is queried from the distributed file system based on the temporary table.

6. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the data query method of the distributed file system as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data query method of the distributed file system as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data query method of the distributed file system as described in any one of claims 1 to 4.

Images