Industrial data analysis method, storage medium and execution equipment

Abstract

The invention discloses an industrial data analysis method, a storage medium and execution equipment, and the method remarkably improves the efficiency and accuracy of data analysis through the steps of automatic data acquisition, mapping, preprocessing, aggregation and the like. The system can automatically clean, convert and verify the data, ensure the reliability and consistency of the data, and avoid analysis deviation by repairing the column name of the CSV file. In addition, the invention also provides a visual display function, so that the user can more intuitively understand the data and the analysis result. The method has high flexibility and expandability, supports keys and values of the user-defined hash table, aggregation processing rules and the like, and meets data analysis requirements in different scenes. Generally speaking, the industrial data analysis method provides powerful support for intelligentization and optimization of industrial production, is helpful for enterprises to quickly find laws and trends in data, and provides more accurate basis for decision making.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an industrial data analysis method, a storage medium and an execution device. Background Art

[0002] In the industrial field, with the rapid development of information technology and the continuous improvement of intelligence level, the collection, storage and analysis of industrial data have become increasingly important. As an important resource in the industrial production process, industrial data contains rich production information, equipment status information and quality control information. The effective extraction and analysis of this information is of great significance for optimizing production processes, improving production efficiency, reducing production costs and improving product quality.

[0003] However, the massiveness, diversity, and complexity of industrial data pose great challenges to data analysis and processing. In practical applications, industrial data is usually stored in the form of CSV (Comma-Separated Values) files. Although CSV files have the advantages of simple structure and easy reading and writing, they often have problems such as data redundancy, errors, and inconsistent formats when processing large-scale, multi-field industrial data. These problems not only affect the accuracy and efficiency of data analysis, but may also lead to deviations in analysis results, thereby misleading industrial production decisions.

[0004] Therefore, how to efficiently analyze and process industrial data in CSV files has become an urgent problem to be solved. Traditional data analysis methods often rely on manual processing, which is not only time-consuming and labor-intensive, but also difficult to cope with the processing needs of large-scale data. With the continuous development of computer technology and data processing technology, automated data analysis methods have gradually become a research hotspot. These methods can automatically extract data from CSV files and perform preprocessing, aggregation, and visualization operations, thereby greatly improving the efficiency and accuracy of data analysis. Summary of the invention

[0005] 1. Technical issues to be resolved To solve the above problems, the present invention proposes an industrial data analysis method, storage medium and execution device, aiming to solve the problems of data redundancy, errors, format inconsistency, etc. that often exist when CSV files in the prior art process large-scale, multi-field industrial data.

[0006] (II) Technical solution An industrial data analysis method of the present invention comprises: Get industrial data from the specified CSV file, and map the acquired industrial data into a hash table to get the processing result; Based on the processing result, read the header of the CSV file, repair the column names to obtain the repaired column list; Reads data from a CSV file within a specified time range, aggregates it, and writes the results to the specified output file.

[0007] In the present invention, the step of obtaining industrial data from a specified CSV file and mapping the obtained industrial data into a hash table includes: The key of the hash table is the first field in the CSV file, and the value is the second field.

[0008] In the present invention, the reading of data from a CSV file within a specified time range, aggregating the data, and writing the result to a specified output file includes: Aggregates the data within a specified range of years and writes the results to a CSV file.

[0009] In the present invention, before mapping the acquired industrial data into the hash table, a step of preprocessing the industrial data is also included, wherein the preprocessing includes data cleaning, data format conversion and data verification to ensure the accuracy and consistency of the data mapped into the hash table.

[0010] In the present invention, the keys and values ​​of the hash table can also be customized according to actual needs.

[0011] In the present invention, before the aggregation results are written into the specified output file, a step of post-processing the aggregation results is also included, and the post-processing includes data formatting, data sorting, data screening and data compression, etc., to meet different output requirements.

[0012] The present invention also includes a step of visually displaying the processing results, the repaired list or the aggregation results, so that users can understand and analyze the industrial data more intuitively.

[0013] Another computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the steps of the industrial data analysis method described in the above technical solution.

[0014] Another execution device of the present invention includes an execution device body and a controller, and is characterized in that the controller includes a processor and a computer program stored in a memory and executable on the processor, and when the processor executes the program, the steps of the industrial data analysis method described in the above technical solution are implemented.

[0015] (III) Beneficial effects Compared with the prior art, the present invention has the following beneficial effects: (1) Through automated data acquisition, mapping, preprocessing and aggregation, the present invention can quickly process large-scale, multi-field industrial data, significantly improving the efficiency of data analysis. Compared with traditional manual processing methods, the present invention can save a lot of time and energy and reduce the cost of data analysis.

[0016] (2) During the data processing process, the present invention performs strict data cleaning, format conversion and verification operations to ensure the accuracy and consistency of the data. At the same time, by repairing the column names of the CSV file, the analysis deviation caused by incorrect or inconsistent column names is avoided, further improving the accuracy of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the logical structure of the analysis method; Figure 2 Schematic diagram of the framework structure of the execution equipment.

[0019] 10. Processor, 11. Memory, 12. Communication interface, 13. Communication bus. DETAILED DESCRIPTION

[0020] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1:

[0021] like Figure 1 The industrial data analysis method shown, in the industrial field, with the continuous advancement of informatization and intelligence, the analysis of industrial data has become increasingly important. A large amount of industrial data is usually stored in the form of CSV files. How to efficiently analyze and process these data. The present invention proposes an industrial data analysis method, which can effectively obtain industrial data from CSV files and perform a series of processing, aggregation and visualization to meet the data analysis needs in different scenarios.

[0022] In order to achieve this goal, the industrial data analysis method of the present invention specifically includes the following steps: S100, obtaining industrial data from a specified CSV file, and mapping the obtained industrial data into a hash table to obtain a processing result.

[0023] Specifically, the system will obtain industrial data from the specified CSV file. In this process, the system will read the data in the CSV file line by line and map the data of each line to a hash table. The key of the hash table is set to the first field in the CSV file, and the value is set to the second field. This design makes it easy to quickly retrieve the corresponding data value based on the key.

[0024] Before mapping the data to the hash table, the system will also preprocess the industrial data. The preprocessing steps include data cleaning, data format conversion, and data verification. Data cleaning is mainly to remove redundant, erroneous or invalid information in the data; data format conversion is to convert the data into a format that the system can process; data verification is to ensure the accuracy and consistency of the data. These preprocessing steps are crucial for subsequent data analysis, and they can ensure the reliability and validity of the data mapped to the hash table.

[0025] S200: Based on the processing result, read the header of the CSV file, repair the column names to obtain a repaired column list.

[0026] After acquiring the industrial data, the system will read the header of the CSV file, that is, the column name part. Since the column names of the CSV file may sometimes be wrong or inconsistent, the system will repair the column names and obtain a repaired column name table. This repaired column name table will serve as the basis for subsequent data processing to ensure the accuracy and consistency of data processing.

[0027] S300, read data from a CSV file within a specified time range, perform aggregation, and write the result to a specified output file.

[0028] Next, the system will read the data from the CSV file within the specified time range and perform aggregation processing. Aggregation processing mainly combines the data within the specified year range according to certain rules to obtain more comprehensive data analysis results. In the present invention, the results of the aggregation processing will be written into a new CSV file for subsequent analysis and use.

[0029] Before writing the aggregation results to the output file, the system will also post-process the aggregation results. The post-processing steps include data formatting, data sorting, data filtering, and data compression. Data formatting is to make the data easier to read and understand; data sorting is to facilitate users to view data in a certain order; data filtering is to meet users' specific data analysis needs; data compression is to reduce the size of the output file and improve storage and transmission efficiency.

[0030] In particular, in addition to the above data processing steps, the present invention also provides a visual display function. The system can visualize the processing results, the repaired list or the aggregation results, so that users can understand and analyze industrial data more intuitively. The visual display can be in various forms such as charts and dashboards, which can be customized according to user needs. Embodiment 2:

[0031] An embodiment of the present invention provides a computer-readable storage medium.

[0032] The computer-readable storage medium provided in the embodiment of the present invention stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned industrial data analysis methods can be implemented.

[0033] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0034] For an introduction to the computer-readable storage medium provided in an embodiment of the present invention, please refer to the above method embodiment, and the present invention will not elaborate on it here.

[0035] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0036] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0037] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. Embodiment 3:

[0038] An embodiment of the present invention provides an execution device.

[0039] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of an execution device provided by the present invention, and the execution device may include: Memory for storing computer programs; The processor is used to implement the steps of any one of the above-mentioned industrial data analysis methods when executing a computer program.

[0040] like Figure 2 , which is a schematic diagram of the composition structure of the execution device, the execution device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0041] In the embodiment of the present invention, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices.

[0042] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute the operations in the embodiment of the push button switch fault detection method.

[0043] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present invention, the memory 11 at least stores programs for implementing the following functions: Get industrial data from the specified CSV file, and map the acquired industrial data into a hash table to get the processing result; Based on the processing result, read the header of the CSV file, repair the column names to obtain the repaired column list; Reads data from a CSV file within a specified time range, aggregates it, and writes the results to the specified output file.

[0044] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0045] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0046] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0047] Of course, it should be noted that Figure 2 The structure shown does not constitute a limitation on the execution device in the embodiment of the present invention. In actual applications, the execution device may include Figure 2 More or fewer components than shown, or combinations of certain components.

[0048] Finally, the evaluation results and evaluation content are sent to the end user and the cloud for data storage.

[0049] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.

[0050] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.

[0051] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. An industrial data analysis method, characterized in that: The industrial data analysis method comprises: Get industrial data from the specified CSV file, and map the acquired industrial data into a hash table to get the processing result; Based on the processing result, read the header of the CSV file, repair the column names to obtain the repaired column list; Reads data from a CSV file within a specified time range, aggregates it, and writes the results to the specified output file.

2. The industrial data analysis method according to claim 1, characterized in that: The obtaining of industrial data from a specified CSV file and mapping the obtained industrial data into a hash table includes: The key of the hash table is the first field in the CSV file, and the value is the second field.

3. The industrial data analysis method according to claim 2, characterized in that: The steps of reading data from a CSV file within a specified time range, aggregating the data, and writing the results to a specified output file include: Aggregates the data within a specified range of years and writes the results to a CSV file.

4. The industrial data analysis method according to claim 3, characterized in that: Before mapping the acquired industrial data into the hash table, the process also includes preprocessing the industrial data. The preprocessing includes data cleaning, data format conversion and data verification to ensure the accuracy and consistency of the data mapped into the hash table.

5. The industrial data analysis method according to claim 4, characterized in that: The keys and values ​​of the hash table can also be customized according to actual needs.

6. The industrial data analysis method according to claim 5, characterized in that: Before writing the aggregation results into the specified output file, the step of post-processing the aggregation results is also included. The post-processing includes data formatting, data sorting, data screening and data compression to meet different output requirements.

7. The industrial data analysis method according to claim 6, characterized in that: The method also includes a step of visually displaying the processing results, the repaired column name table or the aggregation results, so that users can understand and analyze the industrial data more intuitively.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the industrial data analysis method described in any one of claims 1 to 7 are implemented.

9. An execution device, comprising an execution device body and a controller, characterized in that: The controller includes a processor and a computer program stored in a memory and executable on the processor. When the processor executes the program, the steps of the industrial data analysis method according to any one of claims 1 to 7 are implemented.

Images