Quick data exploration

Abstract

A computer-implemented method for fast data exploration of data to be uploaded can be provided. The method (100) comprises uploading a first data set from a local system (102), determining that the first data set is not corrupted (104). The method (100) further comprises, in parallel with said uploading: selecting a predefined number of records from the first data set and constructing a second data set (106), determining statistics and metadata about the first data set (108), and visualizing the second data set, the statistics and the metadata (110).

Description

TECHNICAL FIELD

[0001] The present invention relates generally to uploading large amounts of data from a client system to a server, and more specifically to a computer-implemented method for fast data exploration of data to be uploaded. The invention further relates to a data exploration system and a computer program product for fast data exploration of data to be uploaded. BACKGROUND

[0002] Currently, AI (Artificial Intelligence) is ubiquitous in enterprises of any size and many types of enterprise and consumer applications. As a result, a large number of developers are involved in developing or integrating AI technology. As a consequence, and since a large amount of computing resources is usually required to train machine learning models, such services are often implemented as central services or cloud-based services.

[0003] Conventional machine model learning methods are based on machine learning algorithms and use a known set of data samples together with associated annotations in order to train a general model that is able to predict annotations for unseen data samples. For example, a deep learning model for image classification is trained on a set of input images and associated image class labels. Later, during a test phase, the machine learning (ML) model is used to predict image class labels for new unseen images. The input data samples can include a mix of annotated and unannotated data samples that are processed together during the training phase.

[0004] From a general point of view, input data samples are either available as captured input data or they can be generated by a data generator. A data generator can be represented as a real-world sensor (e.g. a still photo camera or a video camera, an audio system), a human (e.g. a text author) or a simulator (an artificial system that aims to produce a certain type of data samples). Conventional data recognition methods can operate with a fixed number of annotated training samples that can form a training set of data samples. These training data sets are usually relatively large.

[0005] Developers of machine learning models usually need to form an understanding of the structure of the available training data in order to select an appropriate machine learning algorithm. Usually, a number of trial-and-error cycles in the form of trial-and-error are required in this phase in order to select an appropriate AI algorithm with associated hyperparameters for a given problem and the available training data.

[0006] Since a large amount of computing power is required during the training session, the training, data inspection, test phase, etc. are done on centralized computing resources while the developer continues to work on his local workstation. However, the large training data sets usually have to be uploaded to the central resources, which can be very time consuming, resulting in unproductive time for the developer.

[0007] There are several publications on computer-implemented methods for fast data exploration of data to be uploaded.

[0008] Document US 8,805,835 B2 discloses systems, methods, and techniques for managing data sets, in particular, using metadata to represent, acquire, and propagate large data sets and data files. Data representation, acquisition, and propagation of large data sets and data files using metadata is described.

[0009] Document US 6,308,168 B1 discloses a data presentation module that represents panels driven by metadata to allow a user to configure the presentation of information from a computer database system. Upon initial installation of the data presentation module software, the data presentation module is customized for the user's environment by configuring metadata tables. Changes to the panels presented by the data presentation module can be performed by changing the metadata tables without modifying the data presentation module software.

[0010] A drawback of the known solutions can be that the transmission or upload of data from a local system to a server continues to require a significant amount of time, thereby reducing the productivity of a data scientist when developing ML models.

[0011] Therefore, it can be desirable to overcome the drawback of not being able to continue working on data while the data is being uploaded, in particular in the context of developing machine learning algorithms. SUMMARY

[0012] According to an aspect of the present invention, a computer-implemented method for fast data exploration of data to be uploaded can be provided. The method can comprise uploading a first data set having a first size from a local system and determining that the first data set is not corrupted. The method can further comprise, in parallel with the uploading, also performing selecting a predefined number of records from the first data set to build a second data set, determining statistics and metadata about the first data set, and visualizing the second data set, the statistics, and the metadata.

[0013] According to another aspect of the present application, a data exploration system for fast data exploration of data to be uploaded can be provided. The system can comprise means for uploading a first data set having a first size from a local system and means for determining that the first data set is not corrupted. Further, the system can comprise means for selecting a predefined number of records from the first data set to build a second data set, means for determining statistics and metadata about the first data set, and means for visualizing the second data set, the statistics and the metadata. Thereby, the means for selecting, the means for determining statistics and metadata and the means for visualizing can operate in parallel in time with the means for uploading.

[0014] The proposed computer-implemented method for fast data exploration of data to be uploaded can provide a number of advantages, technical effects, contributions and / or improvements:

[0015] A data scientist working in an environment comprising a local system, i.e. a local workstation using a thin client, e.g. a web browser, as user interface, and a server system having more computational resources for performing the training of ML models can save a lot of time, be highly productive and can allow a better usage of the available computational resources. Because developing ML models requires many iterations, many trial and error and a lot of experimentation with data, the transfer time or upload time from the local system to the server can significantly reduce the productivity of the data scientist. This is also a result of experimenting with different sets of training data that can have to be uploaded from the local system to the server.

[0016] The proposed concept can allow the data scientist to continue his experiments and data exploration during the upload of the training data from the local system to the server based on a representative subset of the first data set that is in the process of being uploaded from the local system to the server.

[0017] The statistics and the metadata derived from the first data set can help the user or data scientist to get an impression about the selected training data and can establish a basis for configuring additional tools for data cleansing, data normalization, data correction and / or data adoption. The tools so configured can then be applied to the first data set. Thereby, the user or data scientist can not be limited to running the tools on the local system or on the server to which the first data set can have been uploaded.

[0018] The proposed concept can not limit the freedom of the data scientist to experiment and adapt the data before starting the ML training process, but rather gives him more time and flexibility to work with only a subset, in particular the second data set, in order to improve his productivity and in order to be more productive using the available resources, in particular computing resources, as he can work with the second data set which can be representative of the first data set and only needs little resources for fast data exploration.

[0019] In the following, additional embodiments of the inventive concept applicable to the method as well as the system will be described.

[0020] According to an advantageous embodiment of the method, the statistical data comprises at least one of the group comprising minimum values (in particular data values of a column), maximum values, median values, mean values, quartile values, data frequencies, amount of missing data values (e.g. in a particular record or a particular column). It is also possible to determine additional statistical data which go beyond the classical meaning of the metadata of a data set.

[0021] According to another advantageous embodiment of the method, the metadata comprises at least one of the group comprising the number of rows, the number of columns and the column field types. In addition, the metadata can also comprise additional data about the data in the first data set. This can for example also include the size and type of records, the total amount of data and explicit metadata (e.g. added at the beginning or end of the first data set). Such further additional metadata can be extracted from the first data set only.

[0022] According to an allowed embodiment, the method can also comprise sending the statistical data and the metadata in parallel to uploading the first data set. The destination of the sending can be the same destination as for the first data set. The statistical data can or can not be stored locally, i.e. they can also be transferred from the local, client system to the server via a REST (Representational State Transfer) request.

[0023] According to an optional embodiment of the method, determining that the first data set is not corrupted can be performed before the uploading starts. This has the advantage that data corruption can be detected early in the process and thus the maximum probability of an impossible correct transfer. Upon detecting corrupted data, the uploading can be inhibited and a warning to the user can be generated. Thus, if e.g. a checksum can be established to confirm a correct and completed transfer which can not have occurred, a failure can be detected early ("fast failure") and not at the end of the transfer.

[0024] According to a useful embodiment of the method, the destination of the uploading can be a cloud computing based service. In one embodiment, the cloud computing based service can be a machine learning algorithm to be trained with the uploaded first data set.

[0025] According to an advantageous embodiment, the method can further comprise configuring and potentially also executing a tool (e.g. a data modeling tool or another data manipulation tool) by using the statistics, the metadata and / or the second data set in parallel to the upload. Thus, the configuration of the tool can be locally on the client system or remotely on a server system. However, in the second case, it can also be useful to upload the second data set (in particular in parallel to the upload of the first data set) to the server. On the other hand, it can be advantageous to configure the tool locally at the client during the upload and to use the uploaded configuration on the server side. So, the data scientist using the proposed method is not restricted to the system on which the tool is configured and executed.

[0026] Thus, and according to yet another advantageous embodiment, the method can further comprise executing the configured tool on the first data set (in particular on the server). This feature can advantageously be used especially in case the tool is configured on the client side. Moreover, it can be required to transfer the configuration data from the client to the server side tool.

[0027] According to a preferred embodiment of the method, the size of the second data set can be a fraction of the size of the first data set. Typically, the second data set can comprise about 1000 records (or even less). This can represent a typical number to obtain an initial training of a machine learning tool or the idea of a required data modeling in order to train a machine learning system at the server side with a large amount of training data. Moreover, the relation between the second data set and the first state set in terms of the size of the data sets can be in the range of e.g. 1 / 100, 1 / 1000, 1 / 10000, etc. One of the things a data scientist needs to know is that data can be asymmetric; i.e. a prediction about the behavior of a group of people shall be made and the training data shall represent this group of people equally in all its aspects (e.g. gender, race, age, health, education, etc.). Thus, the data scientist needs to make sure that the training data has roughly the same amount of records for each appropriate aspect in order not to risk a one-sided bias of the ML model.

[0028] According to an allowed embodiment of the method, the selection from the first data set can comprise selecting data from the beginning, the middle, the end, randomly or according to a predefined algorithm from the first data set. In particular, this selection process can depend on the type of available data, the machine learning system to be trained, the data source, and the expected bias of the data, and many other parameters.

[0029] Furthermore, embodiments can take the form of a related computer program product, accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. BRIEF DESCRIPTION OF DRAWINGS

[0030] It should be noted that embodiments of the invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise indicated, in addition to any combination of features belonging to one type of subject matter, also any combination between features

[0031] The above-defined aspects and further aspects of the present invention will be apparent from the examples of embodiments to be described hereinafter with reference to the accompanying drawings and are realized by what is described hereinafter and shown in the drawings. The invention is not limited to the embodiments described hereinafter, but can be practiced with variation of their conditions and components. The scope of the invention is defined by the claims.

[0032] Preferred embodiments of the present invention will be described by way of example only with reference to the accompanying drawings, in which:

[0033] Figure 1 A block diagram of an embodiment of a computer-implemented method of the present invention for fast data exploration of data to be uploaded is shown.

[0034] Figure 2 A block diagram of a subset of some technical components for performing the method according to an embodiment is shown.

[0035] Figure 3 A flowchart of an embodiment of the concepts presented here is shown.

[0036] Figure 4 A block diagram of an embodiment of a data exploration system for fast data exploration of data to be uploaded is shown.

[0037] Figure 5 An embodiment of a computing system comprising a data exploration system according to Figure 4 is shown.

[0038] Figure 6 A cloud computing environment according to an embodiment in which at least part of the inventive concept can be deployed is shown. DETAILED DESCRIPTION

[0039] In the context of the present specification, the following conventions, terms and / or expressions can be used:

[0040] The term "fast data exploration" can denote the ability of a data scientist (or any other user) to perform an evaluation with respect to a large (first) data set that is typically used to train an ML-based system based on analyzing a representative, much smaller subset of the large data set. The term can in particular relate to the option of evaluating a representative subset of the first data set using statistical data metadata with respect to the large first data set. The fast data exploration can be performed on a local system of the data scientist in contrast to the requirement of accessing the data on a server-side system. However, the subset, the statistical data and the metadata can also be transmitted from the local system to the server system.

[0041] The term "upload" can denote in the present document the process of sending or transmitting data and in various embodiments the transmission of a large amount of data that is typically stored on a local data storage system to a remote computer system, e.g. a server, in particular a cloud-based computing system that has a large amount of computing resources in comparison to the local system.

[0042] The term "local system" can denote a workstation of a data scientist, a personal computer, a thin client such as a web browser or any other user interface system.

[0043] The term "first data set" can denote a data set that is typically used for machine learning purposes, e.g. training data. In general, the amount of the first data set can be large such that the transmission from the local system to the server-side system can take a large amount of time thereby slowing down the activities of the under-resourced data scientist. In a conventional system, the data scientist can not be able to work on the first data set during the time the first data set is transmitted, i.e. uploaded, from the local system to the server.

[0044] The term "corruption" can denote that the data is not useful, e.g. a wrong checksum at the end of the data transmission, or can not be otherwise transmittable from one computing system to another. It should be understood that a data file can be in a corrupted state before it is used or transmitted. The data file can be in different formats, e.g. CSV (comma separated values). As one example, a CSV file can be corrupted for several reasons, including invalid formatting, use of invalid characters, line breaks within a field, etc.

[0045] The term "second data set" can denote a data set that is much smaller if compared to the first data set. A typical size of the second data set can be in the range of about 1000 records or even less. However, there can also be a fixed relationship between the size of the first data set and the second data set like 1 / 100, 1 / 1000, 1 / 10000, etc. The fixed relationship can be based on a specified sample size. The second data set can be stored on the local system or the second data set can be uploaded to a server. When storing the second data set on the local system, an advantageous feature of the second data set is that it can be processed on the local system with very few resources only. On the other hand, when the second data set is stored on a server system, it is also easy for the local system to remotely access the second data set via a web browser and work with the second data set.

[0046] The term "statistical data" can denote determined and / or calculated information about the first data set. This can include, but is not limited to, the minimum value in a column, the maximum value in a column, the median value in a column, the mean value in a column, the quartile value in a column, the frequency of data in a column, the amount of missing data values, and a range of other statistical data that can go beyond the classical meaning of metadata. The statistical data can include the number of values in the data set that do not comply with the specified format of the data set.

[0047] The term "metadata" can denote information about the data, in particular the first data set. Typically, the metadata is descriptive and relates to structural components or management components of the reference data of the first data set. The metadata can include, for example, the number of rows and columns in the data set and the format of the data set. It can be noted that this traditional form of metadata and statistical data should be distinguished from each other with respect to the first data set.

[0048] In the context of this document, the term "cloud computing" and the equivalent term "cloud computing-based service" can be interpreted as a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of at least five essential characteristics, three service models, and four deployment models.

[0049] The essential characteristics of cloud computing include:

[0050] (i) On-demand self-service. Consumers can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.

[0051] (ii) Broad network access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0052] (iii) Resource pooling. The provider's computing resources are consolidated into a

[0053] (iv) Rapid elasticity. Capabilities can be provisioned and released very quickly

[0054] (v) Measured service. Cloud systems automatically control and optimize resource use by leveraging utilization of resources in an economical and efficient manner, and providing alternatives in case of resource shortage.

[0055] Service models used for cloud computing include:

[0056] (i) Cloud Software as a Service (SaaS). The capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0057] (ii) Cloud Platform as a Service (PaaS). The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

[0058] (iii) Cloud Infrastructure as a Service (IaaS). The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[0059] Deployment models for cloud computing include:

[0060] (i) Private cloud. The cloud infrastructure is operated solely for an organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0061] (ii) Community cloud. The cloud infrastructure is shared by several organizations, and it supports mission-critical enterprise resources. It can be managed by the organizations or a third party and can exist on-premises or off-premises.

[0062] (iii) Public cloud. The cloud infrastructure is made available to general public or a large industry group and is owned by an organization selling cloud services.

[0063] (iv) Hybrid cloud. The cloud infrastructure is a composition of two or more types of cloud (private, community, or public) that remain unique entities but are bound together, creating a hybrid nomenclature as standardization or propriety technologies enable data and application portability.

[0064] It can be noted that cloud software takes full advantage of the cloud paradigm by focusing on statelessness (with exceptions), low coupling, modularity, and semantic interoperability through service orientation.

[0065] In the following, a detailed description of the drawings and preferred embodiments will be given. All instructions in the drawings are schematic. First, a block diagram of an embodiment of the computer-implemented method of the present invention for fast data exploration of data to be uploaded will be given. Thereafter, further embodiments and embodiments of a data exploration system for fast data exploration of data to be uploaded will be described.

[0066] Figure 1A flowchart showing a preferred embodiment of a computer-implemented method 100 for fast data exploration of data to be uploaded. The method 100 comprises uploading (102) from a local system a first data set, in particular a large first data set, having a first size. The uploading can be performed from a local system (e.g. a PC (personal computer), a local workstation or a local server) to a receiving server (which can be a more powerful server having much more computing resources). The uploading can be performed using a thin client (e.g. a web browser running on a PC having relatively small processing and memory resources). Typically, machine learning algorithms (such as the training of an ML model or system) can be run on the receiving server (e.g. the server 214). The receiving server can also be integrated into a larger cloud computing center. The local system can be a client system or just a web browser.

[0067] In addition, the method comprises determining (104) that the first data set is not corrupted. If this is the case, the uploading will not be started (or, if corruption is detected after the uploading has started, the uploading will not be completed) in order to use the effects of the fast failure to save time and resources for other efforts. While the determination 104 that the first data set is not corrupted is shown in Figure 1

[0068] A part of the method 100 will also be performed in parallel to the uploading 102, selecting (106) from the first data set a predefined number of records for establishing a second data set (thus, a much smaller subset of the first data set), determining (108) statistics and metadata about the first data set, and visualizing (110) the second data set, the statistics and the metadata. The last steps can be performed based on server activity (i.e. a request) or local system activity (i.e. sending a command). While the operations 106, 108 and 110 are preferably performed in parallel to the uploading operation 102, it is not necessary that the operations 106, 108 and 110 are performed completely in parallel to the operation 102. For example, the operations 106, 108 and 110 can be started before the uploading operation 102 or completed after the uploading operation 102.

[0069] Figure 2 ​A block diagram 200 showing a subset of the technical components involved in performing the method 100 according to an embodiment is shown. In the ML context, a local system 202 can manage a large data set 208 on a storage system 206, which can be one or more storage devices such as hard drives or flash drives, or can include conventional computer components such as processors, buses, and peripherals in addition to one or more storage devices. The ML algorithm to be trained can reside on a server 214. Remote access from the server 214 to the storage system 206 can be constrained by low performance characteristics (e.g., a large amount of time to complete ML training), and require a relatively large amount of network bandwidth, and as such can not be practical or can be impossible. Thus, it can be necessary to transfer, upload, or send the first data set 208 from the local system 202 to the server 214, as indicated by the dashed line for the large data set 220 in the storage system 218 of the attached server. The large data set 220 is a copy of the first data set 208 on the storage system 206. That is, a typical size of such a first data set 208 can be in the range of 50 GB, 100 GB, or even larger (e.g., image data). In general, image data can be used as a basis for training of ML algorithms, so it will be appreciated that the data amounts mentioned are not unusual. However, the upload from the local system 202 to the server system 214 can take a large amount of time, disadvantageously requiring a data scientist to wait until all of the data has been transferred to the server 214 side, which can take a large amount of time.

[0070] As mentioned, the server 214 can be a more powerful computing system having more computing resources than the local system 202. While the capabilities of the server 214 can be a result of the server having more processing cores, more memory, faster clock speeds, etc. than the local system 202, no particular hardware configuration is required. In different embodiments, the features that distinguish the server 214 from the local system 202 are throughput, response time, processing speed, etc. In particular, the server 214 can be distinguished from the local system 202 based on its ability to train an ML model using the first data set 208 / 220 in a fraction of the time that the local system 202 would take. For example, the server 214 can be able to train an ML model using the first data set 208 / 220 in 5% or 10% of the time that the local system 202 would take. As a particular example, the server 214 can be able to train an ML model using the first data set 208 / 220 in 50 minutes, while the same task would take 500 minutes (8.3 hours) on the local system 202. In other cases, the time savings can not be as great, but even if the server 214 needs 50% or 75% of the time that the local system 202 needs, the time savings will likely be significant and advantageous.

[0071] In various embodiments, the local system 202 has a first configuration of processing cores and memory that is capable of training a machine learning model using a first data set in a first time period, and the server 214 has a second configuration of processing cores and memory that is capable of training the same machine learning model using the same first data set in a second time period. Since the server 214 is a more powerful system than the local system 202, the second time period is less than the first time period.

[0072] According to different embodiments, the data scientist can explore the data locally and optionally configure data modeling or data manipulation tools in parallel with the upload using the time required for the upload.

[0073] The local system 202 can be triggered to send the first data set 208 by the sender / uploader 204 to the server 214, where the first data set is received by the receiver 216. The sender / uploader 204 can include data transmission hardware (e.g., a modem) and data transmission software that implements a data communication protocol (e.g., TCP) and an error detection protocol. Prior to (or concurrently with) the sending, a check can be performed at the local system 202 to determine that the first data set is not corrupted. For example, a tool such as CSVLint can be used to check that the first data set contains the types and columns of values that it should, i.e., an automated tool can be used to check that the file can be parsed and is readable. In the case of corrupted data, they are not sent by the sender / uploader 204. Additionally, a check can be performed at the receiver 216 to determine that the first data set remains uncorrupted after transmission.

[0074] To use the transmission time from the local system 202 to the server 214, in parallel to the upload (i.e. data transfer), a subset (i.e. second data set 210) can be generated from the first data set 208 (e.g. this subset can be a small fraction of the first data set), and statistical data and metadata 212 about the first data set can also be derived and / or determined from the first data set. The second data set 210, the statistical data and metadata 212 can be visualized (e.g. displayed on a display device of the local system 202) to the data scientists in order to give them the opportunity to get a first impression of the data in the first data set. The data scientists can decide to configure a data modeling tool to be applied to the first data set 208 based on the knowledge about the second data set 210 gained from the visualization. The configuration of the data modeling tool and / or the selection of the ML algorithm can be done on the local system 202 or on the server 214. Typically, during this phase, preparatory tasks of data cleansing, data normalization activities are tested on the second data set which can be representative of the first data set. In different embodiments, one or more data cleansing or data normalization software programs can be applied to the second data set on the local system 204 to determine which programs can be most effective and how any parameters of the data cleansing or data normalization software can be most effectively configured.

[0075] Due to the relatively high trial-and-error approach in ML model development, the proposed approach can save the data scientists a lot of time. The access to the first data set 220 on the server side can be a significantly more powerful server with significantly more computational resources than the local system 202 by the ML algorithm executed on the server 214 or another independent ML system 222 (which is similar to the server 214).

[0076] It can also be noted that the ML algorithm can be embedded into a cloud computing environment to which the server 214 and the ML system 222 can belong, such as described in Figure 6 .

[0077] Figure 3A flowchart 300 showing an embodiment of the concepts presented herein. It begins with making a first dataset available on a local system (e.g., local system 202) (302). The process flow then diverges into multiple branches. First, uploading of the first dataset is initiated (304). As an initial step in this activity, it is determined (306) whether the first dataset is corrupt. Operation 306 can be done prior to uploading the first dataset or performed concurrently with uploading the first dataset. Software tools such as CVSLint can be used to determine whether the first dataset is parseable and readable. If the first dataset is corrupt such that there is a risk that the transfer from the local system to the server can not be successful, then uploading is stopped (310). In the second case where the first dataset is not corrupt, uploading continues (308) to completion, which can last a relatively long time due to the large amount of data to be uploaded.

[0078] Second, a second dataset is extracted from the first dataset (312). The selection process can be performed in a manner such that the second dataset (denoted as DS in the figure) is a good representation of the first dataset. For example, to obtain a good representation, the selection process can include random sampling, systematic sampling, stratified sampling, cluster sampling, or a combination of two or more of these techniques. In the case of asymmetric data, the sampling technique can sample the first dataset in a manner such that each group of data having a particular aspect (e.g., gender) is equally or proportionally represented in the second dataset. In different embodiments, extracting (312) the second dataset using the selection process can be performed by the local system 202. Next, metadata (denoted as MD in the figure) and statistics (denoted as SD in the figure) of the first dataset are determined (314). In different embodiments, determining MD and SD (314) can be performed by the local system 202. Then, the statistics, metadata, and second dataset are visualized (316) to, for example, a data scientist for evaluation of the visualized data on a display device. Optionally, the second dataset, as well as the statistics and metadata of the first dataset, can be uploaded (318) to the server (shown in dashed box) and visualized 316 in a web browser at the local system 202. Next, the data scientist can configure (320) a tool based on the evaluation of the visualized data. The tool can be enabled to normalize, clean, correct, etc. the data of the second dataset and the first dataset; in other words, the tool can be a data modeling tool. While in some embodiments, the data modeling tool can run on the local system 202, typically, the data modeling tool runs only on the server 214 and is accessed by the data scientist via a web browser at the local system 202. When using a data modeling tool running on the server 214, the second dataset 210, MD, and SD need to be uploaded to the server 214.

[0079] Optionally, the configuration of the data modeling tool can also be uploaded (322) to the server. Next, the data modeling tool can be applied (324) to the first data set. Thereby, after the upload of the first data set is complete, the data scientist can quickly perform the steps for preparing the smaller second data set on the original large first data set. The data modeling tool can be configured to perform these steps on the first data set in response to a single command, such as "change data set and rerun notebook?". The training (326) of the ML model of the ML system can start using the first data set after the transmission to the server has successfully ended. In different embodiments, the training of the ML model can start after the first data set has been transmitted and after the data cleansing or data normalization of the first data using the configuration parameters determined using the second data set at the local system 202 has been performed. After the training of the ML model is complete, the process can end (332).

[0080] Figure 4 A block diagram of an embodiment of a data exploration system 400 for quick data exploration of data to be uploaded is shown. The system comprises at least a means for uploading a first data set having a first size from a local system (specifically a sender 402 (e.g., sender / upload 204)) and a means for determining that the first data set is uncorrupted (specifically a first determination unit 404), e.g., a software tool implementing a test to determine whether a data file or other data structure is corrupted. Examples include CSVLint and tools that verify checksums. If this is the case, the sender 402 does not become active.

[0081] Furthermore, the data exploration system 400 comprises a means for selecting a predefined number of records from the first data set to build a second data set (specifically a selection unit 406). For example, the selection unit 406 can contain software that samples the data using random sampling, systematic sampling, stratified sampling, cluster sampling, or a combination of two or more of these techniques. The data exploration system 400 also comprises a means for determining statistics and metadata about the first data set (specifically a second determination unit 408). For example, the second determination unit 408 can comprise software needed to compute statistics (statistics are at least one of the group including: minimum value, maximum value, median value, mean value, quartile value, frequency of data, amount of missing data values) and software needed to extract metadata, where metadata is at least one of the group including: number of rows, number of columns, type of column fields. As one example of software for determining statistics or metadata about the first data, the custom or proprietary determination unit 408 can be written in JavaScript for a particular first data set. Furthermore, custom or proprietary code can be used to determine the size, name, last modified time, and type (metadata) of a file using a web browser.

[0082] The data exploration system 400 further comprises means for visualizing the second data set, the statistical data and the metadata, in particular a visualization unit 410. The visualization unit 410 can be any software for visualizing data as e.g. tables, charts, plain text. Examples for software for visualizing data can be proprietary, open source, or commercially available tools. The software for visualizing data can be written in JavaScript and provides for visualizing data within a web browser. In an embodiment, the visualization unit 410 can be a web-based application accessible through a thin client, such as a web browser presenting the data to be visualized in a table format (rows, columns, column names) and a panel format (data overview, variable / column overview). The panel format can include information about individual variables, such as a histogram, mean, median, mode, skew, kurtosis, minimum, maximum, standard deviation of the variable, whether the variable has missing values and if so how many, and a box plot of the variable. For example, the second data set can be visualized in the table format. Further, the first data set can be visualized in the panel format. Thereby, the means for selecting, the means for determining the statistical data and the metadata, and the means for visualizing are operated in parallel in time with the means for uploading.

[0083] It can be noted that the hardware of the data exploration system 400 can implement units or modules (i.e. the transmitter 402, the first determining unit 404, the specific selection unit, the second determining unit 408, and the visualization unit 410) can be electrically connected for data exchange. Alternatively, they can also be connected via an internal bus system of the data exploration system for data and signal exchange.

[0084] Turning to Figure 5 Before Figure 6 a cloud computing environment is shown in which at least part of the inventive concept, in particular the server-side components, can be deployed.

[0085] Figure 6 A cloud computing environment 600 is shown in which at least part of the inventive concept can be deployed. A set of functional abstraction layers provided by the cloud computing environment is shown. It should be understood beforehand that Figure 6 The components, layers and functions shown in the cloud computing environment are only intended to be illustrative and embodiments of the present invention are not limited thereto. As depicted, the following layers and respective functions are provided: a hardware and software layer 602 includes hardware and software components. Examples of hardware components include: mainframes 604; servers 606; RISC (Reduced Instruction Set Computer) architecture based servers 608; blade servers 610; storage devices 612; networks 614 and network components 614. In some embodiments, software components include network application server software 616 and / or database software 618.

[0086] Virtualization layer 620 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 622; virtual storage 624; virtual networks 626, including virtual private networks; virtual applications and operating systems 628; and virtual clients 630. In one example, management layer 632 can provide the functions described below. Resource provisioning 634 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 636 provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 638 provides access to the cloud computing environment for consumers and system administrators. Service level management 640 provides cloud computing resource allocation and management such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment 642 provide pre-arrangement for, and procurement of, cloud computing resources for which future requirements are anticipated in accordance with an SLA.

[0087] Workloads layer 644 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 646; software development and lifecycle management 648; virtual classroom education delivery 650; data analytics processing 652; transaction processing 654; and server-side components of the solutions presented herein 656.

[0088] Embodiments of the application can be implemented with virtually any type of computer, regardless of the platform being suitable for storing and / or executing program code. Figure 5 A computing system 400 suitable for executing program code related to the presented methods is shown by way of example.

[0089] The computing system 500 is only one example of a suitable computer system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the present application described herein, whether or not implementation is associated with computer system 500. In computing system 500, there are a number of components that can be present in various combinations of hardware and / or software. Examples of well- known computing systems, environments, and / or configurations that can be suitable for use with computer system / server 500 include, but are not limited to, personal computers, server computers, thin clients, thick clients, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed cloud computing environments that include any of the above systems or devices, and the like. The computer system / server 500 can be described in the general context of computer system-executable instructions, such as program modules, being executed by the computer system 500. Generally, program modules can include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system / server 500 can be practiced in distributed cloud computing environments with other computer systems coupled

[0090] As shown, computer system / server 500, is operational with numerous other general purpose or special purpose computing system environments. Computer system / server 500 can be any available

[0091] The system memory 504 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 508 and / or cache memory 510. Computer system / server 500 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 512 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a "hard drive"). Although not specifically shown, a magnetic disk drive can also be used for reading and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive can be used for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media. Each of these devices can be connected to bus 506 by one or more data media interfaces. As will be further depicted and described below, memory 504 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0092] Program / utility 516 having a set (at least one) of program modules 516 can be stored in memory 504 by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating systems, one or more application programs, other program modules, and program data or some combination thereof, can include an implementation of a network environment. Program modules 516 generally carry out the functions and / or methodologies of embodiments of the application, as described herein.

[0093] Computer system / server 500 can also communicate with one or more external devices 518 such as a keyboard, a pointing device, a display 520, etc.; one or more devices that enable a user to interact with computer system / server 500; and / or any devices (e.g., network card, modem, etc.) that enable computer system / server 500 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interfaces 514. Still yet, computer system / server 500 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 522. As depicted, network adapter 522 can communicate with the other components of computer system / server 500 via bus 506. It should be understood that although not specifically shown, other hardware and / or software components could be used in conjunction with computer system / server 500. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0094] In addition, the data exploration system 400 for fast data exploration of data to be uploaded can be attached to the bus system 506.

[0095] The description of various embodiments of the present application have been presented for purposes of illustration, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0096] The present application can be embodied as a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0097] The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus). Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and an optical disk. Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read / write (CD-R / W), DVD, and Blu-ray® disks.

[0098] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0099] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for storage and / or execution.

[0100] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0101] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0102] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, that can be accessed by a computer, a programmable data processing apparatus. Such non-transitory computer readable storage medium and / or computer- readable media can include, but is not limited to, at least one of the following: semiconductor memory devices, e.g., RAM, ROM, PROM, EEPROM, and the like; magnetic disk storage media; and optical storage media.

[0103] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0104] The flow diagrams and / or block diagrams in the drawings are illustrations of architectures, functional processes and operations that can be implemented in systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart and / or block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions (‘instruction(s)’). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0105] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0106] The corresponding structure, material, acts, means, or equivalents that carry out the described functions for an element that accompanies, is incorporated in, or is otherwise a part of the described embodiments, are expressly incorporated herein by reference. The description of the present application has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the application in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the application. The embodiment was chosen and described in order to best explain the principles of the application and the practical application, and to enable others skilled in the art to understand the application for various embodiments with various modifications as are suited to the particular use contemplated.

[0107] Briefly, the inventive concept can be summarized in the following clauses: 1、

[0109] 2. A computer-implemented method for fast data exploration of data to be uploaded, the method comprising:

[0110] - uploading a first data set of a first size from a local system,

[0111] - determining that the first data set is not corrupted;

[0112] - further performing in parallel with said uploading

[0113] - selecting from the first data set a predefined number of records to build a second data set,

[0114] - determining statistics and metadata about the first data set, and

[0115] - visualizing the second data set, said statistics and said metadata.

[0116] 3. The method according to clause 1, wherein said statistics comprise at least one of the group comprising minimum, maximum, median, mean, quartile, frequency of data, amount of missing data values.

[0117] 4. The method according to clause 1 or 2, wherein said metadata comprise at least one of the group comprising number of rows, number of columns, type of column fields.

[0118] 5. The method according to any of the preceding clauses, further comprising

[0119] - further sending said statistics and metadata in parallel with the uploading of the first data set.

[0120] 6. The method according to any of the preceding clauses, wherein determining that the first data set is not corrupted is performed before said uploading is started.

[0121] 7. The method according to any of the preceding clauses, wherein the destination of the upload is a cloud computing based service.

[0122] 8. The method according to any of the preceding clauses, further comprising

[0123] - configuring tools by using the statistics, the metadata and / or a second data set in parallel to the upload.

[0124] 9. The method according to clause 7, further comprising

[0125] - executing the configured tools on the first data set.

[0126] 10. The method according to any of the preceding clauses, wherein the size of the second data set is a fraction of the size of the first data set.

[0127] 11. The method according to any of the preceding clauses, wherein the selection from the first data set comprises

[0128] - selecting data from the beginning, the middle, the end, randomly, or according to a predefined algorithm from the first data set.

[0129] 12. A data exploration system for fast data exploration of data to be uploaded, the system comprising - means for uploading a first data set of a first size from a local system,

[0130] - means for determining that the first data set is not corrupted,

[0131] - means for selecting from the first data set to build a second data set of a predefined number of records,

[0132] - means for determining statistics and metadata about the first data set, and

[0133] - means for visualizing the second data set, the statistics and the metadata,

[0134] wherein the means for selecting, the means for determining statistics and metadata and the means for visualizing are operated in parallel in time to the means for uploading.

[0135] 13. The data exploration system according to clause 11, wherein the statistics comprise at least one of the group comprising minimum, maximum, median, mean, quartile, frequency of data, amount of missing data values.

[0136] 14. The data exploration system according to clause 11 or 12, wherein the metadata comprise at least one of the group comprising number of rows, number of columns, type of column fields.

[0137] 15. The data exploration system according to any of clauses 11 to 13, further comprising

[0138] - means for also uploading statistical data and metadata in parallel with the uploading of the first data set.

[0139] 16. The data exploration system according to any of clauses 11 to 14, wherein the means for determining that the first data set is not corrupted is performed before the means for uploading the first data set starts.

[0140] 17. The data exploration system according to any of clauses 11 to 15, wherein the destination of the uploading is a cloud-based service.

[0141] 18. The data exploration system according to any of clauses 11 to 16, further comprising

[0142] - means for configuring tools by using the statistical data, the metadata and / or a second data set in parallel with the uploading.

[0143] 19. The data exploration system according to clause 17, wherein the configured tools are executed on the first data set.

[0144] 20. The data exploration system according to any of clauses 11 to 18, wherein the size of the second data set is a fraction of the size of the first data set.

[0145] 21. A computer program product for fast data exploration of data to be uploaded, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more computing systems or controllers to cause the one or more computing systems to:

[0146] - upload a first data set of a first size from a local system,

[0147] - determine that the first data set is not corrupted;

[0148] - also perform in parallel with the uploading

[0149] - selecting from the first data set a predefined number of records to build a second data set,

[0150] - determining metadata about the first data set, and

[0151] - visualizing the second data set, the statistical data and the metadata.

Claims

1. A computer-implemented method for fast data exploration of data to be uploaded, the method comprising: uploading a first data set of a first size from a local system to a server; determining by the local system that the first data set is not corrupted; and performing in parallel to the uploading: selecting a certain number of records from the first data set and constructing a second data set with the selected records, determining statistics and metadata about the first data set, and visualizing the second data set, the statistics and the metadata.

2. The method of claim 1, wherein the statistics include at least one of a group comprising: minimum, maximum, median, mean, quartile values, frequency of data values, and amount of missing data values.

3. The method of claim 1, wherein the metadata comprises at least one of a group comprising: number of rows, number of columns, and type of column fields.

4. The method of claim 1, further comprising: sending the statistics and metadata to the server, wherein sending the statistics and metadata to the server is performed in parallel to uploading the first data set to the server.

5. The method of claim 1, wherein determining that the first data set is not corrupted is performed before the uploading starts.

6. The method of claim 1, wherein the local system has a first configuration of processing cores and memory that is capable of training a machine learning model using the first data set in a first time period, and the server has a second configuration of processing cores and memory that is capable of training the machine learning model using the first data set in a second time period, wherein the second time period is less than the first time period.

7. The method of claim 1, further comprising configuring a data modeling tool using the statistics, the metadata, or the second data set, wherein the data modeling tool is configured in parallel to uploading the first data set.

8. The method of claim 7, further comprising executing the configured data modeling tool on the first data set.

9. The method of claim 1, wherein the second data set has a second size, and the first size of the first data set is greater than the second size.

10. The method of claim 1, wherein selecting from the first data set comprises: selecting data from the first data set from the beginning, middle, end, randomly, or according to a predefined algorithm.

11. A data exploration system for fast data exploration of data to be uploaded, the system comprising: a local system having a processing unit and a memory; a network adapter configured to upload a first data set of a first size from the local system to a server computer system; a first determination unit configured to determine that the first data set is not corrupted; a selection unit configured to select a certain number of records from the first data set and construct a second data set with the selected records, wherein the selection unit is operable to select the certain number of records and construct the second data set in parallel to uploading the first data set to the server computer system; a second determination unit configured to determine statistics and metadata about the first data set in parallel to uploading the first data set to the server computer system; and a visualization unit configured to visualize the second data set, the statistics and the metadata in parallel to uploading the first data set to the server computer system. minimum, maximum, median, mean, quartile values, frequency of data values, and amount of missing data values. number of rows, number of columns, and type of column fields. sending the statistics and metadata to the server, wherein sending the statistics and metadata to the server is performed in parallel to uploading the first data set to the server.

12. The data exploration system of claim 11, wherein the statistical data comprises at least one of a group comprising: Minimum, maximum, median, mean, quartile, frequency of data values, and amount of missing data values.

13. The data exploration system of claim 11, wherein the metadata comprises at least one of a group comprising: Number of rows, number of columns, and type of column fields.

14. The data exploration system of claim 11, wherein the network adapter is operable to upload the statistics and the metadata to the server computer system in parallel with uploading the first data set to the server computer system.

15. The data exploration system of claim 11, wherein the first determination unit configured to determine that the first data set is uncorrupted determines that the first data set is uncorrupted prior to the start of the uploading of the first data set to the server computer system.

16. The data exploration system of claim 11, wherein the local system has a first configuration of processing cores and memory that is capable of training a machine learning model using the first data set in a first time period, and the server has a second configuration of processing cores and memory that is capable of training the machine learning model using the first data set in a second time period, wherein the second time period is less than the first time period.

17. The data exploration system of claim 11, further comprising configuring a data modeling tool using the statistics, the metadata, or a second data set, wherein the data modeling tool is configured in parallel with uploading the first data set to the server computer system.

18. The data exploration system of claim 17, further comprising executing the configured data modeling tool on the first data set.

19. The data exploration system of claim 11, wherein the second data set has a second size, and a first size of the first data set is greater than the second size.

20. A computer program product for fast data exploration of data to be uploaded, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more computing systems or controllers to cause the one or more computing systems to: upload a first data set of a first size from a local system to a server; determine, by the local system, that the first data set is uncorrupted; and in parallel with the uploading: select a particular number of records from the first data set and construct a second data set with the selected records, determine statistics and metadata about the first data set, and visualize the second data set, the statistics, and the metadata.

Images