Visualization method for data cleaning

Abstract

The invention discloses a visualization method for data cleaning. The method comprises the following steps: S100, visualizing data import; s200, visualization of built-in template matching: matching an original field name in the imported data with a target field name of a pre-configured template, carrying out visualization, if matching succeeds, executing a step S300, and otherwise, executing a step S400; s300, if the matching succeeds, visualization is carried out, and the step S500 is executed; s400, visualization of matching failure is carried out; s500, visualization of user editing is carried out; s600, the visualization of the original data is previewed; s700, operation visualization is selected, the step S800 is executed, otherwise, editing is selected, and the step S900 is executed; s800, visualization of cleaning: displaying and providing a pre-configured cleaning rule, and executing the step SA00; s900, visualization of editing: displaying and providing column operation selection; sA00, the visualization of the cleaned data is previewed; sA01: confirming the visualization of updating and storage: if the updating is confirmed, updating and storing the data, otherwise, returning to edit.

Description

Technical Field

[0001] The present invention belongs to the field of data forensics analysis and visualization, and relates to a visualization method in the forensics process, and particularly to a visualization method for data cleaning. Background Art

[0002] With the development of forensics business, a large amount of Excel data, including data in CSV format, needs to be imported. Due to the differences in user behaviors, production habits, and regional agreements of Excel and CSV data producers, there are many different data formats. For different formats of data, a simple and efficient visualization method (including import and display methods) is required, and the data can be matched and cleaned to improve the efficiency and accuracy of data processing.

[0003] Currently, many in the data forensics analysis industry do not have methods for data cleaning and its visualization. After importing the data, it is directly stored in the database. Changing the data format can only be done in the original data Excel, which is extremely inconvenient, and the display interface is not intuitive and clear.

[0004] In view of the above problems, there is an urgent need to develop a visualization method for data cleaning that is simple to operate, efficient and fast, and has an intuitive display effect. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention provides a visualization method for data cleaning, including the following steps:

[0006] S100: Visualization of data import: Read data in Excel and / or CSV format and visualize the display interface and / or interaction interface of data import.

[0007] S200: Visualization of built-in template matching: Match the original field names in the imported data with the target field names of the pre-configured templates and visualize the display interface and / or interaction interface of the matching. If the match is successful, execute step S300; otherwise, execute step S400.

[0008] S300: Visualization of successful match: Display the result of successful match and visualize the display interface and / or interaction interface of the comparison relationship between the original field and the target field of the template, and execute step S500.

[0009] S400: Visualization of failed match: Display the result of failed match and visualize the display interface and / or interaction interface of the prompt information, where the prompt information includes allowing manual adjustment, reselecting other templates, and saving new templates.

[0010] S500: Visualization of user editing: Display the edited target fields and original fields, and the display interface and / or interaction interface for visualizing data editing, for displaying to the user and / or providing a choice on whether editing is required;

[0011] S600: Visualization of previewing original data: The display interface and / or interaction interface for visualizing the previewed original data, for displaying to the user and / or providing a choice on whether preview is required;

[0012] S700: Visualization of selection operations: The display interface and interaction interface for visualizing the selected operations, for displaying and providing operation choices to the user. If cleaning is selected, execute step S800; otherwise, if editing is selected, execute step S900;

[0013] S800: Visualization of cleaning: Display and provide pre-configured cleaning rules, clean the data and update the data to the cleaned data, and the display interface and / or interaction interface for visualizing the cleaned data and the updated data, and execute step SA00;

[0014] S900: Visualization of editing: Display and provide choices for column operations, and the display interface and interaction interface for visualizing column operations, where column operations include adding columns and splitting columns;

[0015] SA00: Visualization of previewing data after cleaning: The display interface and / or interaction interface for visualizing the previewed data after cleaning, for displaying to the user and / or providing a choice on whether preview is required;

[0016] SA01: Visualization of confirming update and saving: If the update is confirmed, update the data and save it; otherwise, return to editing, and the display interface and interaction interface for visualizing the confirmed update and saving.

[0017] Preferably, step S400 includes the following steps:

[0018] Visualization of allowing manual adjustment: Display and allow the user to make manual adjustments, and the display interface and / or interaction interface for visualizing the user's manual adjustments, or

[0019] Visualization of allowing selection of other templates: Display and allow selection of other templates, and the display interface and / or interaction interface for visualizing the selection of other templates, or

[0020] Saving as a new template: Display and allow the user to save the manually adjusted template as a new template, and the display interface and / or interaction interface for visualizing the saving as a new template.

[0021] Preferably, the cleaning rules include date and time cleaning, data extraction, split extraction, timestamp conversion, special character removal, absolute value conversion, base conversion, conditional replacement, column combination, multiple magnification, row ignoring, and year completion.

[0022] Preferably, the visualization interface is a Web interface.

[0023] Preferably, the method includes the following steps:

[0024] Provide a drag-and-drop interaction interface to allow manual adjustment of the correspondence between the target field and the original field;

[0025] Save the new template and include the new template in the template library for selection in subsequent matching.

[0026] Preferably, step S800 includes: providing graphical controls on the Web interface to receive configuration of user parameters, where the graphical controls include drop-down menus, sliders, and input boxes.

[0027] Preferably, step S900 includes: providing column operation buttons on the Web interface to pop up a visualization configuration window, where the column operation buttons include adding columns and splitting columns.

[0028] The beneficial effects of the present invention are:

[0029] 1. What you see is what you get: After the original data is imported, it will be displayed on this interface. Users can easily operate, and all operations can directly modify the data. Finally, the imported data will be the modified data for storage.

[0030] 2. While taking into account visualization, it greatly improves the intuitiveness, convenience, and efficiency of data cleaning. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is the overall flowchart of the present invention;

[0032] Figure 2 is an example diagram of the display interface and / or interaction interface showing the result of successful matching and visualizing the correspondence between the original field and the target field of the template in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0033] The present invention will be further described below in conjunction with the drawings and embodiments.

[0034] As Figure 1 shown, the method of the present invention includes the following steps:

[0035] S100: Visualization of data import: Read data in Excel and / or CSV format and visualize the display interface and / or interaction interface of data import.

[0036] The visualization interface of this invention application is a Web interface.

[0037] S200: Visualization of built-in template matching: Match the original field names in the imported data with the target field names of the pre-configured templates and visualize the matching display interface and / or interaction interface. If the match is successful, execute step S300; otherwise, execute step S400.

[0038] S300: Visualization of successful match: Display the result of successful match and visualize the comparison relationship between the original field and the target field of the template in the display interface and / or interaction interface, and execute step S500.

[0039] Figure 2 The example diagram of the display interface and / or interaction interface showing the result of successful match and visualizing the comparison relationship between the original field and the target field of the template in the embodiment of the present invention is shown.

[0040] Such as Figure 2 shown, for example, "Name", "Account Number", "Name of the Bank Where the Account is Opened", "Account Opening Date", etc. in the original field are respectively matched with "Customer Name", "Card Number", "Account Opening Branch" and "Account Opening Date" of the target field of the template; in addition, the remaining fields of the original field are exactly the same as the target field of the template. Therefore, in this embodiment, it is regarded as a successful match.

[0041] S400: Visualization of failed match: Display the result of failed match and visualize the display interface and / or interaction interface of the prompt information, where the prompt information includes allowing manual adjustment, reselecting other templates, and saving a new template. Step S400 includes the following steps:

[0042] Visualization of allowing manual adjustment: Display and allow the user to manually adjust, and visualize the display interface and / or interaction interface allowing the user to manually adjust, including the following steps:

[0043] In this embodiment, a drag-and-drop interaction interface is provided to allow manual adjustment of the comparison relationship between the target field and the original field.

[0044] Allow selection of other templates: Display and allow selection of other templates, and visualize the display interface and / or interaction interface allowing selection of other templates.

[0045] Save as a new template: Display and allow the user to save the manually adjusted template as a new template, and visualize the display interface and / or interaction interface allowing saving as a new template.

[0046] In this embodiment, save the new template and include the new template in the template library for selection in subsequent matches.

[0047] S500: Visualization of user editing: Displays the edited target fields and original fields, and the display interface and / or interaction interface for visual data editing, used to display to the user and / or provide a choice on whether editing is required;

[0048] S600: Visualization of previewing original data: The display interface and / or interaction interface for visualizing the previewed original data, used to display to the user and / or provide a choice on whether previewing is required;

[0049] S700: Visualization of selection operations: The display interface and interaction interface for visualizing the selected operations, used to display to the user and provide a choice of operations. If cleaning is selected, step S800 is executed; otherwise, if editing is selected, step S900 is executed;

[0050] S800: Visualization of cleaning: Displays and provides pre-configured cleaning rules, cleans the data and updates the data to the cleaned data, and the display interface and / or interaction interface for visualizing the cleaned data and the updated data, and executes step SA00;

[0051] Specifically, the cleaning rules include date and time cleaning, data extraction, split extraction, timestamp conversion, special character removal, absolute value conversion, base conversion, conditional replacement, column combination, multiple magnification, row ignoring, and year completion.

[0052] Step S800 includes: Providing graphical controls on the Web interface for receiving the configuration of user parameters, where the graphical controls include drop-down menus, sliders, and input boxes.

[0053] S900: Visualization of editing: Displays and provides a choice of column operations, and the display interface and interaction interface for visualizing column operations, where the column operations include adding columns and splitting columns;

[0054] Step S900 includes: Providing column operation buttons on the Web interface for popping up a visual configuration window, where the column operation buttons include adding columns and splitting columns.

[0055] SA00: Visualization of previewing the cleaned data: The display interface and / or interaction interface for visualizing the previewed cleaned data, used to display to the user and / or provide a choice on whether previewing is required;

[0056] SA01: Visualization of confirming update and saving: If the update is confirmed, the data is updated and saved; otherwise, return to editing, and the display interface and interaction interface for visualizing the confirmed update and saving.

[0057] Through the method provided by the present invention, the technical problem in the prior art that there is no visualization method for data cleaning is solved.

[0058] It should be understood that the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or modifications can be made according to the above description, and all such improvements and modifications shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A visualization method for data cleaning, characterized in that Including the following steps: S100: Visualization of data import: Read data in Excel and / or CSV format and visualize the display interface and / or interactive interface of data import; S200: Visualization of built-in template matching: Match the original field names in the imported data with the target field names of the pre-configured templates and visualize the display interface and / or interactive interface of the matching. If the match is successful, execute step S300; otherwise, execute step S400; S300: Visualization of successful match: Display the result of successful match and visualize the display interface and / or interactive interface of the comparison relationship between the original field and the target field of the template, and execute step S500; S4 S400: Visualization of failed match: Display the result of failed match and visualize the display interface and / or interactive interface of the prompt information, where the prompt information includes allowing manual adjustment, reselecting other templates, and saving new templates; S500: Visualization of user editing: Display the edited target fields and original fields and visualize the display interface and / or interactive interface of data editing to display and / or provide the user with the option of whether to edit; S600: Visualization of previewing original data: Visualize the display interface and / or interactive interface of the previewed original data to display and / or provide the user with the option of whether to preview; S700: Visualization of selection operation: Visualize the display interface and interactive interface of the selected operation to display and provide the user with operation options. If "cleaning" is selected, execute step S800; otherwise, if "editing" is selected, execute step S900; S800: Visualization of cleaning: Display and provide the pre-configured cleaning rules, clean the data and update the data to the cleaned data, and visualize the display interface and / or interactive interface of the cleaned data and the updated data, and execute step SA00; S900: Visualization of editing: Display and provide column operation options and visualize the display interface and interactive interface of column operations, where column operations include adding columns and splitting columns; SA00: Visualization of previewing cleaned data: Visualize the display interface and / or interactive interface of the previewed cleaned data to display and / or provide the user with the option of whether to preview; SA01: Visualization of confirming update and saving: If the update is confirmed, update the data and save it; otherwise, return to editing and visualize the display interface and interactive interface of the confirmed update and saving.

2. A visualization method for data cleaning according to claim 1, characterized in that, Step S400 includes the following steps: Visualization of allowing manual adjustment: Display and allow the user to make manual adjustments, and visualize the display interface and / or interactive interface of the allowed user manual adjustments, or Visualization of allowing selection of other templates: Display and allow selection of other templates, and visualize the display interface and / or interactive interface of the allowed selection of other templates, or Visualization of saving as a new template: Display and allow the user to save the manually adjusted template as a new template, and visualize the display interface and / or interactive interface of the allowed saving as a new template.

3. A visualization method for data cleaning according to claim 1, characterized in that, The cleaning rules include date and time cleaning, data extraction, split extraction, timestamp conversion, special character removal, absolute value conversion, base conversion, conditional replacement, column combination, multiple magnification, row ignoring, and year completion.

4. A visualization method for data cleaning according to claim 1, characterized in that, The visualization interface is a Web interface.

5. A visualization method for data cleaning according to claim 2, characterized in that, It includes the following steps: Provide a drag-and-drop interaction interface that allows manual adjustment of the correspondence between target fields and original fields; Save the new template and include the new template in the template library for selection in subsequent matching.

6. A visualization method for data cleaning according to claim 1, characterized in that, Step S800 includes: providing graphical controls on the Web interface for receiving user parameter configurations, where the graphical controls include dropdown menus, sliders, and input boxes.

7. A visualization method for data cleaning according to claim 1, characterized in that Step S900 includes: providing column operation buttons on the Web interface for popping up a visualization configuration window, where the column operation buttons include adding columns and splitting columns.

Images