Data acquisition and processing method and system in industrial pilot plant test platform tool

Abstract

The invention relates to the technical field of data processing, and discloses a data acquisition and processing method in an industrial pilot plant test platform tool, which comprises the following steps: acquiring industrial pilot plant test data to obtain acquired data; preprocessing the collected data to obtain preprocessed data, and identifying a data class center of the preprocessed data; analyzing an optimal attribute split point of the preprocessed data to identify a spatial local deviation rate of the preprocessed data, performing data sorting on the preprocessed data to obtain distributed data, identifying spatial access points of the distributed data, constructing a data grid of the preprocessed data based on the spatial access points, and utilizing the data grid to obtain the distributed data. Performing data screening on the distributed data to obtain screened data; and identifying a data structure of the screened data, performing classification structure processing on the screened data, and performing data encryption protection on the structure processing data to obtain target data. According to the invention, the rationality of data management in industrial pilot plant test platform tools can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data collection and processing method and system in an industrial pilot platform tool. Background Art

[0002] Industrial pilot platform tools play a key role in promoting industrial innovation and development, and the data collection and processing methods are crucial. Building an efficient data collection and processing system applied to industrial pilot platform tools can help companies accurately record, analyze and manage various types of data in the pilot process, thereby improving the quality and efficiency of industrial pilots. At the same time, technicians can monitor the changes in pilot parameters in real time and conduct in-depth data analysis, and adjust the pilot plan in time to provide more accurate pilot services. In addition, the system can also facilitate information sharing and communication between different departments, between personnel in different technical positions, and between enterprises and external partners, thereby reducing pilot risks and improving team collaboration efficiency.

[0003] At present, data collection and processing in industrial pilot platform tools are generally achieved through traditional data collection equipment and software to build a system with basic data storage and simple analysis functions to achieve pilot data management. However, with this method, many data collection and processing tasks require manual intervention. Due to the large amount of multi-source heterogeneous data in the industrial pilot process, users are not accurate enough in data collection and processing when the data volume is large, resulting in unreasonable data management in industrial pilot platform tools. Summary of the invention

[0004] The present invention provides a data collection and processing method and system in an industrial pilot platform tool, the main purpose of which is to improve the rationality of data management in the industrial pilot platform tool.

[0005] To achieve the above-mentioned purpose, the present invention provides a data collection and processing method in an industrial pilot platform tool, comprising:

[0006] Obtaining data collection requirements of the industrial pilot platform tool, and collecting industrial pilot data according to the data collection requirements to obtain collected data;

[0007] Performing data preprocessing on the collected data to obtain preprocessed data, identifying data features of the preprocessed data, calculating feature averages of the data features, and determining a data class center of the preprocessed data based on the feature averages;

[0008] Analyzing the best attribute splitting point of the preprocessed data by using the data class center, identifying the spatial local deviation rate of the preprocessed data based on the best attribute splitting point, sorting the preprocessed data based on the spatial local deviation rate to obtain distributed data, identifying the spatial access point of the distributed data, constructing a data grid of the preprocessed data based on the spatial access point, and filtering the distributed data by using the data grid to obtain filtered data;

[0009] Identify the data structure of the screening data, perform classification structure processing on the screening data based on the data structure to obtain structure processing data, perform data encryption protection on the structure processing data to obtain security data, perform data storage on the security data to obtain target data.

[0010] Optionally, calculating the feature average of the data feature includes:

[0011] Construct a data set corresponding to the data feature, and based on the data set, calculate the numerical feature average of the data feature using the following formula:

[0012]

[0013] Among them, α represents the feature average, N represents the number of data sets, and x uy Represents the u-th data feature of the y-th data set.

[0014] Optionally, determining the data class center of the preprocessed data according to the feature average value includes:

[0015] Determining an initial class center of the preprocessed data based on the feature average value;

[0016] Allocating data points to the initial class center to obtain an allocated class center;

[0017] Updating the data of the distribution class center to obtain the target data class center;

[0018] Performing a center evaluation on the target data class center to obtain an evaluation result;

[0019] When the evaluation result meets the preset result, the target data class center is used as the data class center of the preprocessed data.

[0020] Optionally, analyzing the optimal attribute splitting point of the pre-processed data by using the data class center includes:

[0021] Based on the data class center, determining the attribute evaluation index of the preprocessed data;

[0022] According to the attribute evaluation index, calculate the index value of each attribute of the preprocessed data under different values;

[0023] Based on the indicator value, an optimal attribute splitting point of the preprocessed data is determined.

[0024] Optionally, the identifying the spatial local deviation rate of the preprocessed data based on the optimal attribute splitting point includes:

[0025] Based on the optimal attribute splitting point, calculating the similarity of adjacent data points in the preprocessed data;

[0026] Based on the similarity, the spatial local deviation rate of the preprocessed data is calculated using the following formula:

[0027]

[0028] Among them, SLDF(d i ) represents the spatial local deviation rate, d j represents the jth data point in the preprocessed data, d i represents the i-th data point in the preprocessed data, N(d i ) represents the data point d i The set of neighborhood data points, w ij represents the similarity between data points i and j, m represents the number of spatial dimensions of preprocessed data, x ik represents the eigenvalue of data point i in the kth spatial dimension, x jk Represents the eigenvalue of data point j in the kth spatial dimension.

[0029] Optionally, based on the data structure, performing classification structure processing on the screening data to obtain structure-processed data includes:

[0030] Based on the data structure, the screening data is divided into structured data and unstructured data;

[0031] Constructing a relational data table of the structural data;

[0032] Using the relational data table to perform data integration on the structure data to obtain integrated data;

[0033] Classifying and processing the unstructured data to obtain classified processed data;

[0034] The integrated data and the classified processed data are merged to obtain structure processed data.

[0035] Optionally, the classifying and processing the unstructured data to obtain the classified processed data includes:

[0036] Identifying text data, image data, and audio data in the unstructured data;

[0037] Performing natural language processing on the text data to obtain processed text data;

[0038] Performing image processing on the image data to obtain image processed data;

[0039] Performing audio processing on the audio data to obtain audio processing data;

[0040] The processed text data, the image processed data and the audio processed data are fused to obtain classified processed data.

[0041] Optionally, the performing data encryption protection on the structure processing data to obtain secure data includes:

[0042] identifying data sensitivity of data processed by the structure;

[0043] Based on the data sensitivity, user definition is performed on the access role corresponding to the structure processing data to obtain a defined user;

[0044] querying the structure for independent attributes of the processed data;

[0045] Associating the independent attribute with the defined user to obtain data-role information;

[0046] Setting permissions for the stored data based on the data-role information to obtain permission data;

[0047] The authority data is encrypted to obtain secure data.

[0048] Optionally, encrypting the authority data to obtain security data includes:

[0049] Calculating the data characteristic value of the permission data to obtain the permission data characteristic value;

[0050] Based on the characteristic value of the authority data, the authority data is encrypted using the following formula to obtain security data:

[0051]

[0052] Among them, E represents security data, XOR(.) represents exclusive OR operation, T e represents the permission data feature value of the e-th permission data, M represents the feature dimension of the permission data, Represents the intermediate summation result after linear processing of all feature dimensions of all permission data.

[0053] The embodiment of the present invention obtains target data by storing the security data, which can improve data access efficiency and management capabilities, and provide a reliable data source for subsequent data analysis and application. The security data can be stored by utilizing the data repository in the industrial pilot platform tool.

[0054] In order to solve the above problems, the present invention also provides a data acquisition and processing system in an industrial pilot platform tool, the system comprising:

[0055] A data collection module is used to obtain the data collection requirements of the industrial pilot platform tool, so as to collect industrial pilot data according to the data collection requirements and obtain collected data;

[0056] A data class center identification module is used to perform data preprocessing on the collected data to obtain preprocessed data, identify data features of the preprocessed data, calculate feature averages of the data features, and determine the data class center of the preprocessed data based on the feature averages;

[0057] A data screening module, used to analyze the best attribute splitting point of the pre-processed data by using the data class center, identify the spatial local deviation rate of the pre-processed data based on the best attribute splitting point, sort the pre-processed data based on the spatial local deviation rate to obtain distributed data, identify the spatial access point of the distributed data, construct a data grid of the pre-processed data based on the spatial access point, and use the data grid to screen the distributed data to obtain screened data;

[0058] A data storage module is used to identify the data structure of the screening data, classify and structure the screening data based on the data structure to obtain structure processing data, encrypt and protect the structure processing data to obtain security data, and store the security data to obtain target data.

[0059] The present invention obtains the data collection requirements of the industrial pilot platform tool, collects industrial pilot data according to the data collection requirements, and obtains the collected data to clarify the target and direction of data collection, avoid blind data collection, improve the validity and practicality of data, ensure that the collected data is consistent with the actual needs of the industrial pilot platform, and then provide the original data basis for subsequent data processing and analysis; and the collected data is preprocessed to obtain the preprocessed data, which can remove noise, abnormal values ​​and other interference factors, improve the quality of data, and in the industrial pilot data, there may be various errors and irregular data, which can make the data more accurate and reliable through preprocessing; further, the present invention can facilitate the classification of data according to specific attributes by using the data class center, analyzing the optimal attribute splitting point of the preprocessed data, which helps to find the potential patterns and laws in the data, and can provide more fine-grained information for subsequent data analysis and processing; further, the present invention can improve the access efficiency and management ability of data by storing the security data, and obtain the target data, which provides a reliable data source for subsequent data analysis and application. Therefore, the present invention can improve the rationality of data management in the industrial pilot platform tool. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A flow chart of a data collection and processing method in an industrial pilot platform tool provided by an embodiment of the present invention;

[0061] Figure 2 A functional module diagram of a data acquisition and processing system in an industrial pilot platform tool provided by an embodiment of the present invention;

[0062] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0064] The embodiment of the present application provides a data collection and processing method in an industrial pilot platform tool. The execution subject of the data collection and processing method in the industrial pilot platform tool includes but is not limited to at least one of the electronic devices such as the server, the terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the data collection and processing method in the industrial pilot platform tool can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0065] Reference Figure 1 FIG. 1 is a flow chart of a data collection and processing method in an industrial pilot platform tool provided by an embodiment of the present invention. In this embodiment, the data collection and processing method in the industrial pilot platform tool includes:

[0066] S1. Obtain data collection requirements of the industrial pilot platform tool, and collect industrial pilot data according to the data collection requirements to obtain collected data.

[0067] The embodiment of the present invention obtains the data collection requirements of the industrial pilot platform tool to collect industrial pilot data according to the data collection requirements. The collected data can clarify the goal and direction of data collection, avoid blind data collection, improve the validity and practicality of data, ensure that the collected data is consistent with the actual needs of the industrial pilot platform, and provide the original data basis for subsequent data processing and analysis.

[0068] Among them, the industrial pilot platform tool refers to a comprehensive platform and a set of related tools built in the process of industrial development in order to realize the transition of scientific and technological achievements from the laboratory to industrialization. The data collection demand refers to the data generated in the industrial pilot platform tool, such as physical parameters, chemical parameters, electrical parameters and other data requirements.

[0069] Optionally, the data collection requirements can be obtained by obtaining the data requirements of relevant staff of the industrial pilot platform, such as engineers, researchers and managers, to understand their specific data requirements during the pilot process, such as which parameters of data are needed, data accuracy requirements, data collection frequency, etc. The industrial pilot data can be collected through data analysis tools, such as Excel.

[0070] S2. Perform data preprocessing on the collected data to obtain preprocessed data, identify data features of the preprocessed data, calculate feature averages of the data features, and determine a data class center of the preprocessed data based on the feature averages.

[0071] The embodiment of the present invention performs data preprocessing on the collected data to obtain preprocessed data, which can remove interference factors such as noise and outliers, thereby improving the quality of the data. In industrial pilot data, there may be various errors and non-standard data, and preprocessing can make the data more accurate and reliable.

[0072] The pre-processed data can be obtained by performing data cleaning, deduplication and outlier processing on the collected data.

[0073] Furthermore, the embodiments of the present invention can help users understand the distribution and characteristics of data by identifying the data features of the pre-processed data, thereby providing a basis for subsequent data analysis and processing.

[0074] Optionally, the data features of the preprocessed data may be obtained by identifying attribute values ​​of data tags of the preprocessed data.

[0075] Among them, the data characteristics refer to the unique attributes of the data in the industrial pilot platform tool, such as numerical type (such as temperature, pressure, size, etc.), category type (such as product type, equipment status, etc.) and other related characteristics.

[0076] Furthermore, the embodiment of the present invention can quickly understand the overall central position of the group of data features by calculating the feature average value of the data features, thereby improving data processing efficiency.

[0077] As an embodiment of the present invention, the calculating the feature average value of the data feature includes: constructing a data set corresponding to the data feature, and based on the data set, calculating the feature average value of the data feature using the following formula:

[0078]

[0079] Among them, α represents the feature average, N represents the number of data sets, and x uy Represents the u-th data feature of the y-th data set.

[0080] Optionally, the data set may construct different sets, such as numerical sets, category sets, etc., through attributes corresponding to different data features.

[0081] Furthermore, the embodiment of the present invention can perform preliminary classification and summary of the data by determining the data class center of the preprocessed data based on the feature average value, thereby providing a basis for further analysis.

[0082] Among them, the data class center refers to data with special performance in a certain data feature. For example, there are three types of product type features: A, B, and C. Through statistics, it is found that type A appears the most times in the data set, so type A can be used as the class center representative of the product type feature.

[0083] As an embodiment of the present invention, determining the data class center of the preprocessed data according to the feature average value includes: determining the initial class center of the preprocessed data based on the feature average value, assigning data points to the initial class center to obtain an assigned class center, updating data on the assigned class center to obtain a target data class center, performing center evaluation on the target data class center to obtain an evaluation result, and when the evaluation result meets a preset result, using the target data class center as the data class center of the preprocessed data.

[0084] Optionally, the initial class center of the preprocessed data can be obtained by selecting the data point closest to the feature average value, the allocated class center can be obtained by calculating the distance between each data point and its respective class center and assigning it to the class to which the closest class center belongs, the target data class center can be obtained by calculating the average value of each data point in the allocated class center on each feature, as the new class center, and the center evaluation of the target data class center can be performed by calculating the intra-class distance and inter-class distance of the target data class center to evaluate the quality of the class center, wherein the smaller the intra-class distance and the larger the inter-class distance, the better the effect of the class center, and the preset result can be set to that the intra-class distance is not greater than 0.2 and the inter-class distance is not less than 0.8, and can also be set according to the actual application scenario.

[0085] S3. Analyze the optimal attribute splitting point of the preprocessed data by using the data class center, identify the spatial local deviation rate of the preprocessed data based on the optimal attribute splitting point, sort the preprocessed data based on the spatial local deviation rate to obtain distributed data, identify the spatial access points of the distributed data, construct a data grid of the preprocessed data based on the spatial access points, and use the data grid to filter the distributed data to obtain filtered data.

[0086] The embodiment of the present invention utilizes the data class center to analyze the optimal attribute splitting point of the preprocessed data, which can facilitate the classification of data according to specific attributes, help discover potential patterns and laws in the data, and provide more fine-grained information for subsequent data analysis and processing.

[0087] The optimal attribute splitting point refers to a point constructed when classifying data and is used to divide the data set.

[0088] As an embodiment of the present invention, the use of the data class center to analyze the optimal attribute splitting point of the preprocessed data includes: determining an attribute evaluation index of the preprocessed data based on the data class center, calculating the index value of each attribute of the preprocessed data under different values ​​according to the attribute evaluation index, and determining the optimal attribute splitting point of the preprocessed data based on the index value.

[0089] The attribute evaluation index refers to a standard used to measure the effectiveness of each attribute (feature) in the preprocessed data for data partitioning. Common attribute evaluation indicators include information gain, Gini index, etc. The index value refers to the specific value calculated for each attribute in the preprocessed data under different values ​​according to the selected attribute evaluation index.

[0090] Optionally, the attribute evaluation index of the pre-processed data based on the data class center can be determined by analyzing the characteristics of the data class center and the nature and needs of the pre-processed data. If the data class center is relatively clear and the classification of data is more important, you can consider selecting information gain as the attribute evaluation index, because information gain can better measure the contribution of attributes to classification. If you pay more attention to the purity of the data and the balance of the division, you can choose the Gini index. For example, in the industrial pilot platform tool, if the pre-processed data is about the operating parameters of different equipment, and you want to classify the equipment status according to these parameters, you can choose information gain. information gain as an attribute evaluation index to determine which parameters are most helpful in distinguishing different device states; according to the attribute evaluation index, calculating the index value of each attribute of the preprocessed data at different values ​​can be done by dividing the preprocessed data into two parts, calculating the information entropy or Gini index of each part respectively, and then calculating the index value according to the information entropy or Gini index before and after the division; based on the index value, determining the best attribute splitting point of the preprocessed data can be done by comparing the index values ​​of all attributes at different values, and selecting the attribute and value with the best index value (such as the largest information gain or the smallest Gini index) as the best attribute splitting point.

[0091] Furthermore, the embodiment of the present invention can discover abnormal points or outliers in the data by identifying the spatial local deviation rate of the preprocessed data based on the optimal attribute splitting point. These outliers may represent errors, abnormal situations or potentially valuable information in the data. By identifying the spatial local deviation rate, the distribution and characteristics of the data can be better understood.

[0092] The spatial local deviation rate is an indicator used to measure the degree of deviation of each data point in the preprocessed data in the subspace to which it belongs.

[0093] As an embodiment of the present invention, the identifying the spatial local deviation rate of the preprocessed data based on the optimal attribute splitting point includes: calculating the similarity of adjacent data points in the preprocessed data based on the optimal attribute splitting point, and calculating the spatial local deviation rate of the preprocessed data based on the similarity using the following formula:

[0094]

[0095] Among them, SLDF(d i ) represents the spatial local deviation rate, d j represents the jth data point in the preprocessed data, d i represents the i-th data point in the preprocessed data, N(d i ) represents the data point d i The set of neighborhood data points, w ij represents the similarity between data points i and j, m represents the number of spatial dimensions of preprocessed data, x ik represents the eigenvalue of data point i in the kth spatial dimension, x jk Represents the eigenvalue of data point j in the kth spatial dimension.

[0096] Optionally, the calculating of the similarity of adjacent data points in the preprocessed data based on the optimal attribute splitting point may be performed by using the optimal attribute splitting point to determine attribute values ​​in the preprocessed data, and calculating attribute weights of the attribute values, calculating the attribute weights, and determining the similarity.

[0097] The higher the weight, the higher the similarity.

[0098] In the embodiment of the present invention, the pre-processed data is sorted based on the spatial local deviation rate to obtain distributed data, which can more intuitively show the distribution of data and the difference in deviation degree. This helps to further analyze the characteristics of the data and determine the data area that needs to be focused on.

[0099] Optionally, the data sorting of the preprocessed data based on the spatial local deviation rate can be performed by integrating the local deviation rates of the data points in the preprocessed data in each feature dimension to obtain a comprehensive spatial local deviation rate, for example, taking the average or weighted average of the local deviation rates of each dimension as the comprehensive deviation rate, and sorting the preprocessed data according to the comprehensive deviation rate. The sorting can be performed from small to large or from large to small, depending on the analysis requirements.

[0100] The embodiment of the present invention can determine key locations or nodes that can be accessed and operated in the distributed data by identifying the spatial access points of the distributed data.

[0101] Among them, the spatial access point refers to the position of the key data point determined from the distributed data according to specific rules. The key data points can be selected from the distributed data as the spatial access points according to certain rules, such as selecting a specific proportion of data points with a larger or smaller local deviation rate, or selecting representative points in the data distribution.

[0102] Furthermore, the embodiment of the present invention can help users better organize and manage data by constructing a data grid of the pre-processed data based on the spatial access point, making the spatial distribution of data clearer. At the same time, data can also be quickly located and queried through the data grid, thereby improving the efficiency of data processing.

[0103] Optionally, constructing the data grid of the pre-processed data based on the spatial access points can be done by arbitrarily selecting a point that has not been visited among the spatial access points, using a set dimensional interval distance value as a grid basic unit size, and constructing the data grid of the pre-processed data based on the grid basic unit size.

[0104] Furthermore, the embodiment of the present invention can remove unnecessary data or select a specific data subset by filtering the distributed data to obtain filtered data, thereby extracting valuable data.

[0105] Optionally, the data can be filtered by setting data filtering conditions according to specific analysis requirements, such as selecting data in a specific grid unit, or selecting data points that meet a certain deviation rate range for filtering.

[0106] S4. Identify the data structure of the screening data, perform classification structure processing on the screening data based on the data structure to obtain structure processing data, perform data encryption protection on the structure processing data to obtain security data, perform data storage on the security data to obtain target data.

[0107] The embodiment of the present invention can identify the data structure by identifying the data structure of the screening data, thereby better understanding the organization mode and characteristics of the data, and is more conducive to data analysis.

[0108] The data structure refers to the way computers store and organize data. It refers to a collection of data elements that have one or more specific relationships with each other. The data structure includes the logical structure and physical structure of the data.

[0109] Optionally, the data structure of the screened data can be determined by analyzing the characteristics of the data: observing the element types of the data, the relationship between the elements, the organization of the data, and other characteristics to determine the data structure type.

[0110] Furthermore, the embodiment of the present invention can organize and store the data according to a specific structure by performing classification structure processing on the screening data based on the data structure to obtain structure-processed data, so that data management is more orderly and efficient. In this way, operations such as data query, update and deletion can be conveniently performed, thereby improving the speed and accuracy of data processing.

[0111] As an embodiment of the present invention, based on the data structure, the filtered data is classified and structurally processed to obtain structurally processed data, including: based on the data structure, the filtered data is divided into structural data and unstructured data, a relational data table of the structural data is constructed, the structural data is integrated using the relational data table to obtain integrated data, the unstructured data is classified and processed to obtain classified processed data, and the integrated data and the classified processed data are merged to obtain structurally processed data.

[0112] Among them, the structured data refers to data with a clear data structure and format, which can usually be presented in a table form, with each column representing a specific attribute and each row representing a data record, such as table data in a database, data in a spreadsheet, etc. The unstructured data refers to data without a fixed structure or format, which is difficult to represent in a traditional table form, such as images, audio, video, etc. The data relationship table refers to a table form representation used to describe the relationship between each data element in the structured data. For example, in the industrial pilot platform tool, an electronic product with product model A was tested on August 1, 2024. At that time, the temperature was 25°C, the voltage was 5V, and the current was 2A. This record will be reflected as a row of data in the data relationship table. If the product is tested again on August 2, new data will also be added to the table. For products of different models, such as model B, there will also be their own test records in the table. Through this data relationship table, the relationship between various performance parameters of different product models at different time points can be clearly seen.

[0113] Optionally, the relational data table of the structural data can be constructed by using a spreadsheet tool such as Excel, and obtaining the integrated data can be achieved by normalizing the structural data.

[0114] Further, as an optional embodiment of the present invention, the classification processing of the unstructured data to obtain classified processed data includes: identifying text data, image data and audio data in the unstructured data, performing natural language processing on the text data to obtain processed text data, performing image processing on the image data to obtain image processed data, performing audio processing on the audio data to obtain audio processed data, and fusing the processed text data, the image processed data and the audio processed data to obtain classified processed data.

[0115] Among them, the text data refers to information represented in text form, such as books, articles, web pages, etc., the image data refers to information represented in image form, such as photos, graphics, charts, etc., the audio data refers to information represented in sound form, such as music, voice, sound effects, etc., and the data fusion refers to the fusion of data represented by multiple structures into a comprehensive data representation, which can be achieved by inputting the data into the same relational database.

[0116] Optionally, the process of performing natural language processing on the text data to obtain processed text data is: dividing the text data into words, performing text translation on the words to obtain processed text data, performing image processing on the image data to obtain image processing data, performing image processing on the image data using a deep learning model, performing audio processing on the audio data to obtain audio processing data, performing audio feature extraction on the audio data using a deep learning model to obtain audio features, and performing text translation on the audio features.

[0117] Furthermore, the embodiment of the present invention can protect the security and privacy of the data and prevent the data from being illegally accessed and tampered with by performing data encryption protection on the structure processing data to obtain secure data.

[0118] As an embodiment of the present invention, the data encryption protection of the structure processing data to obtain secure data includes: identifying the data sensitivity of the structure processing data, based on the data sensitivity, user-defining the access role corresponding to the structure processing data to obtain the defined user, querying the independent attributes of the structure processing data, associating the independent attributes with the defined user to obtain data-role information, setting permissions for the stored data based on the data-role information to obtain permission data, and encrypting the permission data to obtain secure data.

[0119] Among them, the data sensitivity refers to the importance and sensitivity of the data to the organization or individual. For example, in the process of industrial pilot testing, if cost accounting, investment evaluation and other links are involved, financial information will be generated. For example, the pilot project budget, actual expenditure details, expected returns and other data of a new product. These financial information are crucial to the enterprise and are highly sensitive data. The independent attribute refers to a characteristic of the data, that is, it does not depend on other data or external factors, and can be determined or calculated independently of other data. For example, in the industrial pilot platform for electronic products, the product model is an independent attribute. The product model is usually determined by the manufacturer during the product design stage, and it does not depend on other performance parameters or test results of the product.

[0120] Optionally, data sensitivity can be obtained by classifying the sensitivity of the structure processing data, such as

[0121] In a material industry pilot platform, the following types of data are stored: physical property data such as color and hardness of materials can be divided into one category, indicating low sensitivity; some more conventional parts of material production costs and production process parameters can be divided into two categories, indicating general sensitivity; performance test results of materials under specific environments are divided into three categories, indicating medium sensitivity; core formula data of materials are the fourth category, indicating relatively sensitive; special material data under development for specific high-end fields are the fifth category, indicating high sensitivity. The defined user can be defined by defining the data access administrator corresponding to the structure processing data, such as low-sensitivity data can be accessed by market research and sales personnel; the permission setting of the stored data based on the data-role information to obtain permission data can be achieved by setting the access level of the access role.

[0122] Further, as an optional embodiment of the present invention, encrypting the permission data to obtain security data includes: calculating a data characteristic value of the permission data to obtain a permission data characteristic value, and based on the permission data characteristic value, encrypting the permission data using the following formula to obtain security data:

[0123]

[0124] Among them, E represents security data, XOR(.) represents exclusive OR operation, T e represents the permission data feature value of the e-th permission data, M represents the feature dimension of the permission data, Represents the intermediate summation result after linear processing of all feature dimensions of all permission data.

[0125] The embodiment of the present invention obtains target data by storing the security data, which can improve data access efficiency and management capabilities, and provide a reliable data source for subsequent data analysis and application. The security data can be stored by utilizing the data repository in the industrial pilot platform tool.

[0126] like Figure 2 The figure shows a functional module diagram of a data acquisition and processing system in an industrial pilot platform tool provided by an embodiment of the present invention.

[0127] The data acquisition and processing system 200 in the industrial pilot platform tool described in the present invention can be installed in an electronic device. According to the functions implemented, the data acquisition and processing system 200 in the industrial pilot platform tool described in the present invention may include a data acquisition module 201, a data class center identification module 202, a data screening module 203 and a data storage module 204. The module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0128] In this embodiment, the functions of each module / unit are as follows:

[0129] The data collection module 201 is used to obtain the data collection requirements of the industrial pilot platform tool, so as to collect industrial pilot data according to the data collection requirements and obtain collected data;

[0130] The data class center identification module 202 is used to perform data preprocessing on the collected data to obtain preprocessed data, identify data features of the preprocessed data, calculate feature averages of the data features, and determine the data class center of the preprocessed data based on the feature averages;

[0131] The data screening module 203 is used to analyze the best attribute splitting point of the pre-processed data by using the data class center, identify the spatial local deviation rate of the pre-processed data based on the best attribute splitting point, sort the pre-processed data based on the spatial local deviation rate to obtain distributed data, identify the spatial access point of the distributed data, construct a data grid of the pre-processed data based on the spatial access point, and use the data grid to screen the distributed data to obtain screened data;

[0132] The data storage module 204 is used to identify the data structure of the screening data, and based on the data structure, perform classification structure processing on the screening data to obtain structure processing data, perform data encryption protection on the structure processing data to obtain security data, and perform data storage on the security data to obtain target data.

[0133] In detail, the modules described in the data collection and processing system 200 in the industrial pilot platform tool described in the embodiment of the present invention adopt the same technical means as the data collection and processing method in the industrial pilot platform tool described in the accompanying drawings when used, and can produce the same technical effects, which will not be repeated here.

[0134] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0135] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A data collection and processing method in an industrial pilot platform tool, characterized in that: The method comprises: Obtaining data collection requirements of the industrial pilot platform tool, and collecting industrial pilot data according to the data collection requirements to obtain collected data; Performing data preprocessing on the collected data to obtain preprocessed data, identifying data features of the preprocessed data, calculating feature averages of the data features, and determining a data class center of the preprocessed data based on the feature averages; Analyzing the best attribute splitting point of the preprocessed data by using the data class center, identifying the spatial local deviation rate of the preprocessed data based on the best attribute splitting point, sorting the preprocessed data based on the spatial local deviation rate to obtain distributed data, identifying the spatial access point of the distributed data, constructing a data grid of the preprocessed data based on the spatial access point, and filtering the distributed data by using the data grid to obtain filtered data; Identify the data structure of the screening data, perform classification structure processing on the screening data based on the data structure to obtain structure processing data, perform data encryption protection on the structure processing data to obtain security data, perform data storage on the security data to obtain target data.

2. The data collection and processing method in the industrial pilot platform tool according to claim 1, characterized in that: The calculating the feature average of the data feature includes: Construct a data set corresponding to the data feature, and based on the data set, calculate the numerical feature average of the data feature using the following formula: Among them, α represents the feature average, N represents the number of data sets, and x uy Represents the u-th data feature of the y-th data set.

3. The data collection and processing method in the industrial pilot platform tool according to claim 1, characterized in that: Determining the data class center of the preprocessed data according to the feature average value includes: Determining an initial class center of the preprocessed data based on the feature average value; Allocating data points to the initial class center to obtain an allocated class center; Updating the data of the distribution class center to obtain the target data class center; Performing a center evaluation on the target data class center to obtain an evaluation result; When the evaluation result meets the preset result, the target data class center is used as the data class center of the preprocessed data.

4. The data collection and processing method in the industrial pilot platform tool according to claim 1, characterized in that: The using the data class center to analyze the optimal attribute splitting point of the pre-processed data includes: Based on the data class center, determining the attribute evaluation index of the preprocessed data; According to the attribute evaluation index, calculate the index value of each attribute of the preprocessed data under different values; Based on the indicator value, an optimal attribute splitting point of the preprocessed data is determined.

5. The data collection and processing method in the industrial pilot platform tool according to claim 1, characterized in that: The splitting point based on the optimal attribute includes: Based on the optimal attribute splitting point, calculating the similarity of adjacent data points in the preprocessed data; Based on the similarity, the spatial local deviation rate of the preprocessed data is calculated using the following formula: Among them, SLDF(d i ) represents the spatial local deviation rate, d j represents the jth data point in the preprocessed data, d i represents the i-th data point in the preprocessed data, N(d i ) represents the data point d i The set of neighborhood data points, w ij represents the similarity between data points i and j, m represents the number of spatial dimensions of preprocessed data, x ik represents the eigenvalue of data point i in the kth spatial dimension, x jk Represents the eigenvalue of data point j in the kth spatial dimension.

6. The data collection and processing method in the industrial pilot platform tool according to claim 1, characterized in that: The step of performing classification structure processing on the screening data based on the data structure to obtain structure-processed data includes: Based on the data structure, the screening data is divided into structured data and unstructured data; Constructing a relational data table of the structural data; Using the relational data table to perform data integration on the structure data to obtain integrated data; Classifying and processing the unstructured data to obtain classified processed data; The integrated data and the classified processed data are merged to obtain structure processed data.

7. The data collection and processing method in the industrial pilot platform tool according to claim 6, characterized in that: The classifying and processing the unstructured data to obtain the classified processed data includes: Identifying text data, image data, and audio data in the unstructured data; Performing natural language processing on the text data to obtain processed text data; Performing image processing on the image data to obtain image processed data; Performing audio processing on the audio data to obtain audio processing data; The processed text data, the image processed data and the audio processed data are fused to obtain classified processed data.

8. The data collection and processing method in the industrial pilot platform tool according to claim 1, characterized in that: The step of performing data encryption protection on the structure processing data to obtain secure data includes: identifying data sensitivity of data processed by the structure; Based on the data sensitivity, user definition is performed on the access role corresponding to the structure processing data to obtain a defined user; querying the structure for independent attributes of the processed data; Associating the independent attribute with the defined user to obtain data-role information; Setting permissions for the stored data based on the data-role information to obtain permission data; The authority data is encrypted to obtain secure data.

9. The data collection and processing method in the industrial pilot platform tool according to claim 8, characterized in that: The step of encrypting the authority data to obtain security data includes: Calculating the data characteristic value of the permission data to obtain the permission data characteristic value; Based on the characteristic value of the authority data, the authority data is encrypted using the following formula to obtain security data: Among them, E represents security data, XOR(.) represents exclusive OR operation, T e represents the permission data feature value of the e-th permission data, M represents the feature dimension of the permission data, Represents the intermediate summation result after linear processing of all feature dimensions of all permission data.

10. A data collection and processing system in an industrial pilot platform tool, characterized in that: Used to execute the data collection and processing method in an industrial pilot platform tool as described in any one of claims 1 to 9, the system comprises: A data collection module is used to obtain the data collection requirements of the industrial pilot platform tool, so as to collect industrial pilot data according to the data collection requirements and obtain collected data; A data class center identification module is used to perform data preprocessing on the collected data to obtain preprocessed data, identify data features of the preprocessed data, calculate feature averages of the data features, and determine the data class center of the preprocessed data based on the feature averages; A data screening module, used to analyze the best attribute splitting point of the pre-processed data by using the data class center, identify the spatial local deviation rate of the pre-processed data based on the best attribute splitting point, sort the pre-processed data based on the spatial local deviation rate to obtain distributed data, identify the spatial access point of the distributed data, construct a data grid of the pre-processed data based on the spatial access point, and use the data grid to screen the distributed data to obtain screened data; A data storage module is used to identify the data structure of the screening data, classify and structure the screening data based on the data structure to obtain structure processing data, encrypt and protect the structure processing data to obtain security data, and store the security data to obtain target data.

Images