Excel-based distributed computing method, system, device and medium

Abstract

The present application discloses a distributed computing method, device, equipment and medium based on Excel, wherein the method obtains a data source to be queried; obtains query information input by a user, and parses the query information to obtain information of a dependent business table corresponding to the query information; generates a reading strategy, and reads the data of the dependent business table from the data source; performs distributed computing on the data of the dependent business table to obtain data information queried by the user in the data of the dependent business table. Compared with the related art, the present application integrates distributed computing and multiple heterogeneous data readers on the basis of Excel, so that Excel can directly process multiple heterogeneous data while improving its computing power and achieving faster real-time processing speed.

Description

Technical Field

[0001] The present application relates to the technical field of data analysis and processing, and in particular to a distributed computing method, device, equipment and medium based on Excel. Background Art

[0002] Currently, Excel is widely used in the field of data analysis and business processing for its intuitive and easy-to-use table calculation capabilities, but traditional Excel has some limitations when dealing with complex and large-scale data computing needs. Especially in processing multi-source heterogeneous data and distributed computing, Excel's native functions are difficult to support efficient calculation and real-time processing of large-scale data. Summary of the invention

[0003] The purpose of this application is to provide a distributed computing method, device, equipment and medium based on Excel to solve the problem in the related art that Excel is difficult to perform real-time and efficient calculations on large-scale data.

[0004] In order to achieve the above objectives, this application provides the following technical solutions:

[0005] In a first aspect, the present application provides a distributed computing method based on Excel, comprising:

[0006] Get the data source to be queried;

[0007] Obtaining query information input by the user, and parsing the query information to obtain information of the dependent business table corresponding to the query information;

[0008] Generate a read strategy, and read the data of the dependent business table from the data source;

[0009] Distributed computing is performed on the data in the dependent business table to obtain data information queried by the user in the data in the dependent business table.

[0010] Further, the obtaining of the data source to be queried includes:

[0011] The data source or the data source link is imported into Excel to obtain the data source to be queried.

[0012] Furthermore, the method also includes: saving the data information queried by the user and presenting it to the user.

[0013] Furthermore, generating a reading strategy and reading the data of the dependent business table from the data source includes: generating a reading strategy based on the type of the data source, index information and data size; and reading the data of the dependent business table from the data source according to the reading strategy.

[0014] Furthermore, the type of the data source includes: relational data or non-relational data.

[0015] Furthermore, the performing distributed computing on the data in the dependent business table to obtain the data information queried by the user in the data in the dependent business table includes:

[0016] The data of the dependent business table is split into multiple computing tasks, and different computing nodes used by each computing task are parallelly computed to obtain data information queried by the user in the data of the dependent business table.

[0017] In a second aspect, the present application further provides a distributed computing system based on Excel, comprising:

[0018] The acquisition module is used to obtain the data source to be queried;

[0019] A parsing module is used to obtain the query information input by the user, and parse the query information to obtain the name of the dependent business table corresponding to the query information;

[0020] A reading module, used for generating a reading strategy and reading the data of the dependent business table from the data source;

[0021] The distributed computing module is used to perform distributed computing on the data in the dependent business table to obtain the data information queried by the user in the data in the dependent business table.

[0022] Furthermore, the reading module is specifically used for:

[0023] Generate a read strategy based on the type of the data source, index information, and data volume;

[0024] According to the reading strategy, the data of the dependent business table is read from the data source.

[0025] In a third aspect, the present application further provides a computer electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned Excel-based distributed computing methods when executing the computer program.

[0026] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the Excel-based distributed computing methods described above are implemented.

[0027] The present application provides a distributed computing method, device, equipment and medium based on Excel, wherein the method obtains the data source to be queried; obtains the query information input by the user, and parses the query information to obtain the information of the dependent business table corresponding to the query information; generates a reading strategy, and reads the data of the dependent business table from the data source; performs distributed computing on the data of the dependent business table to obtain the data information queried by the user in the data of the dependent business table. Compared with the related art, the present application integrates distributed computing and multiple heterogeneous data readers on the basis of Excel, so that Excel can directly process multiple heterogeneous data while improving the computing power of Excel and achieving faster real-time processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of a distributed computing method based on Excel in an embodiment of the present application;

[0029] Figure 2 It is a structural schematic diagram of a distributed computing device based on Excel in an embodiment of the present application;

[0030] Figure 3 It is a structural schematic diagram of a computer electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0033] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0034] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0035] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms "a", "said" and "the" used in one or more embodiments of the present application are also intended to include plural forms, unless the context clearly indicates other meanings.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0037] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when the time of".

[0038] At present, although there are some form calculation tools that support distributed computing on the market, these tools still have many shortcomings in terms of supporting the richness of Excel functions, access to heterogeneous data sources, and real-time data processing performance.

[0039] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes are not repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0040] Please refer to Figure 1 , an Excel-based distributed computing method provided in an embodiment of the present application, the method is applied in Excel, and at least comprises the following steps:

[0041] S10. Obtain the data source to be queried.

[0042] Specifically, in this embodiment, it is first necessary to obtain the data source to be queried.

[0043] In a certain embodiment of the present application, step S10, obtaining the data source to be queried, includes: importing the data source or the data source link into Excel, thereby obtaining the data source to be queried.

[0044] It is understandable that the user can import the data source or the link of the data source into Excel, and the computing system can automatically obtain the above data source.

[0045] It should be noted that the above Excel integrates a distributed computing interface and a variety of heterogeneous data reading interfaces.

[0046] S20: Acquire query information input by the user, and parse the query information to obtain information of the dependent business table corresponding to the query information.

[0047] Specifically, in this embodiment, it is necessary to further obtain the user's query information, and then parse the user's query information to obtain the information of the corresponding dependent business table in the data source that the user needs to query.

[0048] It is understandable that the user can enter the name of the business table on the front-end page and set the conditions to be queried. For example, the user can enter the name of the business table on the front-end page and set the query conditions to query the field information of the sales performance in the sales performance table.

[0049] S30: Generate a reading strategy, and read the data of the dependent business table from the data source.

[0050] In one embodiment of the present application, step S30, generating a read strategy and reading data of the dependent business table from the data source, includes:

[0051] S301: Generate a reading strategy according to the type of the data source, index information, and the amount of data.

[0052] S302: Read the data of the dependent business table from the data source according to the reading strategy.

[0053] Specifically, in this embodiment, a reading strategy can be generated according to the type of data source, index information, and the size of the data volume. Exemplarily, different data readers can be selected according to the type of data source, where the type of data source includes relational data and non-relational data, for example, strategies for selecting readers of different databases such as mysql, mongodb, and redis; secondly, strategies for selecting full read, real-time incremental merge read, etc. can also be selected according to the data source index situation of the data source, for example, and thirdly, whether to enable parallel read strategy can be selected according to the size of the data source data volume, etc.

[0054] It should be noted that the read strategy can select a single strategy or a combination of multiple strategies.

[0055] S40: Perform distributed computing on the data in the dependent business table to obtain data information queried by the user in the data in the dependent business table.

[0056] In one embodiment of the present application, the step S40 of performing distributed computing on the data in the dependent business table to obtain the data information queried by the user in the data in the dependent business table includes:

[0057] The data of the dependent business table is split into multiple computing tasks, and different nodes used by each computing task are parallelly computed to obtain data information queried by the user in the data of the dependent business table.

[0058] It can be understood that in this example, the data dependent on the business table is split into multiple computing tasks, and then different computing nodes are used for parallel computing, which can greatly speed up the computing speed.

[0059] It should be noted that, in this example, the calculation process supports the calculation of Excel functions. For example, an Excel function can be set in a cell in the business data table, so that the calculation function of the Excel function can be directly used.

[0060] In one embodiment of the present application, the method further includes: S50, saving the data information queried by the user and presenting it to the user.

[0061] It is understandable that after the calculation is completed, the data can be saved and presented to the user.

[0062] The application provides a distributed computing method based on Excel, which obtains the data source to be queried; obtains the query information input by the user, and parses the query information to obtain the information of the dependent business table corresponding to the query information; generates a reading strategy, and reads the data of the dependent business table from the data source; performs distributed computing on the data of the dependent business table to obtain the data information queried by the user in the data of the dependent business table. Compared with the related art, the application integrates distributed computing and multiple heterogeneous data readers on the basis of Excel, so that Excel can directly process multiple heterogeneous data while improving the computing power of Excel and achieving faster real-time processing speed.

[0063] See also Figure 2 The present application also provides an Excel-based distributed computing system 200, comprising:

[0064] The acquisition module 201 is used to acquire the data source to be queried;

[0065] The parsing module 202 is used to obtain the query information input by the user, and parse the query information to obtain the name of the dependent business table corresponding to the query information;

[0066] A reading module 203, used to generate a reading strategy and read the data of the dependent business table from the data source;

[0067] The distributed computing module 204 is used to perform distributed computing on the data in the dependent business table to obtain data information queried by the user in the data in the dependent business table.

[0068] In one embodiment of the present application, the reading module is specifically used for:

[0069] Generate a read strategy based on the type of the data source, index information, and data volume;

[0070] According to the reading strategy, the data of the dependent business table is read from the data source.

[0071] See also Figure 3 The embodiment of the present application further provides a computer electronic device 300, including a memory 303 and a processor 302, wherein the memory 303 stores a computer program, and when the processor executes the computer program, the steps of any of the above-mentioned Excel-based distributed computing methods are implemented.

[0072] Specifically, the electronic device 300 includes: a transceiver 301, a bus interface and a processor 302, wherein the processor 302 is used to obtain a data source to be queried; obtain query information input by a user, and parse the query information to obtain information of a dependent business table corresponding to the query information; generate a reading strategy, and read the data of the dependent business table from the data source; perform distributed computing on the data of the dependent business table to obtain data information queried by the user in the data of the dependent business table.

[0073] In the embodiment of the present application, the electronic device 300 further includes: a memory 303. Figure 3 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 302 and memory represented by memory 303. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 301 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 302 is responsible for managing the bus architecture and general processing, and the memory 303 may store data used by the processor 302 when performing operations.

[0074] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned Excel-based distributed computing methods are implemented.

[0075] In this embodiment, the computer readable storage medium may be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include but is not limited to: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0076] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0077] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0078] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0079] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0080] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0081] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A distributed computing method based on Excel, characterized in that: include: Get the data source to be queried; Obtaining query information input by the user, and parsing the query information to obtain information of the dependent business table corresponding to the query information; Generate a read strategy, and read the data of the dependent business table from the data source; Distributed computing is performed on the data in the dependent business table to obtain data information queried by the user in the data in the dependent business table.

2. The Excel-based distributed computing method according to claim 1, characterized in that: The step of obtaining the data source to be queried includes: The data source or the data source link is imported into Excel to obtain the data source to be queried.

3. The Excel-based distributed computing method according to claim 1, characterized in that: The method further comprises: The data information queried by the user is saved and displayed to the user.

4. The Excel-based distributed computing method according to claim 1, characterized in that: The generating of the reading strategy and reading the data of the dependent business table from the data source includes: Generate a read strategy based on the type of the data source, index information, and data volume; According to the reading strategy, the data of the dependent business table is read from the data source.

5. The Excel-based distributed computing method according to claim 4, characterized in that: The type of the data source includes: relational data or non-relational data.

6. The Excel-based distributed computing method according to claim 1, characterized in that: The performing distributed computing on the data in the dependent business table to obtain data information queried by the user in the data in the dependent business table includes: The data of the dependent business table is split into multiple computing tasks, and different computing nodes used by each computing task are parallelly computed to obtain data information queried by the user in the data of the dependent business table.

7. A distributed computing system based on Excel, characterized in that: include: The acquisition module is used to obtain the data source to be queried; A parsing module is used to obtain the query information input by the user, and parse the query information to obtain the name of the dependent business table corresponding to the query information; A reading module, used for generating a reading strategy and reading the data of the dependent business table from the data source; The distributed computing module is used to perform distributed computing on the data in the dependent business table to obtain the data information queried by the user in the data in the dependent business table.

8. The distributed computing system according to claim 7, characterized in that: The reading module is specifically used for: Generate a read strategy based on the type of the data source, index information, and data volume; According to the reading strategy, the data of the dependent business table is read from the data source.

9. A computer electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the Excel-based distributed computing method described in any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the Excel-based distributed computing method described in any one of claims 1 to 6 are implemented.

Images