Industrial data analysis method and system based on dynamic form

Abstract

The invention discloses an industrial data analysis method and system based on a dynamic form, and relates to the technical field of data analysis, when a plurality of systems share data, the systems calculate the similarity Xsd of data items through a repeatability analysis module, and screen out repeated data sets based on the similarity, so that the systems can effectively identify and eliminate redundant data, and the data processing efficiency is improved. And unnecessary repeated acquisition and storage are avoided, so that data storage is optimized, storage resources are saved, and the burden of redundant data is reduced. Through a data usage analysis module, the system can identify features of each data item in different contexts within a duplicated data set. Through data context vectorization and analysis of the use difference degree Cyd in different contexts, the system can further judge which data items have different meanings in different systems or service scenes, so that non-redundant data which should not be regarded as redundancy is screened out, and the analysis ensures that the system performs judgment not only based on the data similarity.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a method and system for analyzing industrial data based on dynamic forms. Background Art

[0002] In the wide application of dynamic form technology, it has become a tool that can collect and manage data efficiently and flexibly, especially in the fields of data collection, information management, and intelligent decision-making.

[0003] Although the industrial data analysis system based on dynamic forms has played an important role in improving the efficiency of data display and collection, the current implementation still has several shortcomings and deficiencies. First, redundant data may be generated during data sharing, especially when sharing data between multiple business systems. Due to the different data collection methods and requirements of each system, the same monitoring data may be repeatedly collected and stored. This redundant data not only takes up storage space, but also may affect the efficiency and accuracy of subsequent data analysis. Secondly, the data items in the repeated data group may have different contexts, that is, the same data item has different meanings in different systems or different scenarios. At present, many systems lack effective redundancy identification and data usage analysis in the process of data sharing, and often confuse duplicate data with redundant data. It is difficult to accurately judge which data items are redundant and which data are indispensable even if they are duplicate data.

[0004] The occurrence of the above shortcomings and deficiencies has led to multiple negative effects. In the presence of redundant data, first of all, it will lead to a waste of storage resources, especially in the scenario of large-scale industrial data collection, the storage cost of redundant data is quite huge. Secondly, due to the duplication of data and the lack of effective screening, it may cause data conflicts or deviations in the subsequent data analysis process, reduce the validity of the data and the accuracy of the decision, and may even affect the timeliness of production decisions and equipment maintenance. In the case of failure to accurately handle context differences, data between different business systems may be mistakenly considered to be duplicated or redundant, resulting in data loss or misuse, and ultimately affecting the overall performance of the system and the effectiveness of business operations. For example, some data items may be key decision factors in some scenarios, but due to the failure to correctly identify their context, they may eventually be misjudged as redundant data because they belong to duplicate data, and eventually be eliminated, affecting subsequent production scheduling or quality control. Therefore, the correct identification of redundant data, effective screening, and clear analysis of context are the key to improving the performance and accuracy of industrial data analysis systems. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method and system for analyzing industrial data based on dynamic forms, which solves the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an industrial data analysis system based on dynamic forms, including a form construction module, a repeatability analysis module, a data usage analysis module, a redundancy identification module and a data processing module;

[0007] The form construction module is used to obtain monitoring data of each business system in the industrial park using monitoring instruments, generate dynamic forms based on the monitoring data of each business system, and define conditions for the dynamic forms to dynamically display data items X under corresponding conditions;

[0008] The repetitive analysis module is used to analyze the repetitiveness between business systems when multiple business systems share data, according to the data item X corresponding to the shared data, to calculate and obtain the similarity Xsd, and to filter the repeated data group based on the similarity Xsd;

[0009] The data usage analysis module is used to identify the features of each data item X in the repeated data group in the context according to the repeated data group, and perform data context vectorization to analyze the difference degree of usage of each data item X in the repeated data group in different contexts to obtain the difference degree Cyd, and preliminarily screen out some non-redundant data groups from the repeated data group based on the difference degree Cyd;

[0010] The redundant identification module is used to obtain relevant numerical change data based on the monitoring interval on the basis of the data usage analysis module to calculate the change value Bhz, and combine the trained data recognition model and similarity Xsd, and after dimensionless processing, fit the output judgment index Pdzs;

[0011] The data processing module is used to identify and filter out non-redundant data groups from repeated data groups again according to the determination index Pdzs, and compress and store the redundant data groups.

[0012] Preferably, the form construction module includes a data monitoring unit and a form definition unit;

[0013] The data monitoring unit is used to provide an interface to exchange data with monitoring instruments in each business system to obtain monitoring data of each business system in the industrial park, and to clean the monitoring data of each business system and convert it into a unified format;

[0014] The form definition unit is used to generate a dynamic form based on the monitoring data of each business system acquired and processed by the data monitoring unit, wherein the dynamic form includes the monitoring data of each business system, and the conditions and rules of the dynamic form are defined to display the required related data items X through the defined conditions, and automatically hide the remaining data, wherein the condition and rule definitions include state conditions, user selection conditions, time and date conditions, data item enable and disable conditions, data range verification and data format verification, wherein the data item X refers to the data value collected by each sensor under the defined conditions.

[0015] Preferably, the repeatability analysis module includes a preparation unit, a repeating unit and a judgment unit;

[0016] The preparation unit is used to determine the data items X involved in the sharing of the multiple business systems according to the sharing requirements of the multiple business systems when multiple business systems are sharing data, generate shared data after aggregation, delete the data items X other than the shared data from the monitoring data of each business system based on the shared data, and define the shared data area by using conditions and rules;

[0017] The repeating unit is used to analyze the repetitiveness between the business systems according to the data item X involved in the sharing of multiple business systems displayed in the shared data area, so as to calculate and obtain the similarity Xsd. The similarity Xsd is obtained by the following formula: ;

[0018] In the formula, and Vector representation of data items from two different business systems. and They are vectors and Model.

[0019] Preferably, the judgment unit is used for a preset threshold, and compares the similarity Xsd with the preset threshold to screen the duplicate data group, and the specific screening content is as follows:

[0020] If the similarity Xsd exceeds a preset threshold, it means and If there is similarity between them, the corresponding data item is determined as a duplicate data item and is not stored in the shared data area temporarily. After statistics, a duplicate data group is generated;

[0021] If the similarity Xsd does not exceed the preset threshold, it means and There is no similarity between them yet, so the data items in each business system are stored in the shared data area.

[0022] Preferably, the data usage analysis module includes a vectorization unit, a difference unit and a screening unit;

[0023] The vectorization unit is used to identify, based on the repeated data group, the specific business function role of each data item X in the repeated data group in the corresponding business system, the upstream and downstream relationship of the data item, and the change and fluctuation of the data item in the time dimension, wherein the specific business function role includes sensor data, control instructions, system status and alarm signals, and is represented by classification labels; the upstream and downstream relationship of the data item refers to whether the data item X has input or output of a decision or operation; the change and fluctuation of the data item in the time dimension refers to the time characteristics of the data item X, including the rate of change and the amplitude of fluctuation, which are represented as time series features; the specific business function role of each data item X in the repeated data group in the corresponding business system, the upstream and downstream relationship of the data item, and the change and fluctuation of the data item in the time dimension are vectorized to generate a feature vector Y of each data item in the repeated data group.

[0024] Preferably, the difference unit is used to analyze the difference degree of use of each data item X in the repeated data group in different contexts according to the characteristic vector Y of each data item in the repeated data group to obtain the difference degree Cyd, and the difference degree Cyd is obtained by the following formula: ;

[0025] Where n represents the number of data items in the repeated data group, i=1, 2, ..., n, and The feature vector of the i-th data item in the repeated data group comes from two different business systems.

[0026] Preferably, the screening unit is used to pre-set a difference threshold, and compare and analyze the difference threshold with the difference degree Cyd, so as to preliminarily screen out some non-redundant data groups from the repeated data groups. The specific contents are as follows:

[0027] If the difference degree Cyd exceeds the difference threshold, it means that the context difference of the corresponding data item is in an abnormal state. At this time, the corresponding data item is recorded as a non-redundant data item and counted to obtain a non-redundant data group;

[0028] If the difference degree Cyd does not exceed the difference threshold, it means that the context difference of the corresponding data item is in a normal state, and the corresponding data item is recorded as a redundant data item.

[0029] Preferably, the redundancy identification module includes a significant difference unit and a comprehensive analysis unit;

[0030] The obvious difference unit is used to re-update the repeated data group according to the screening unit, and on the basis of the re-updated repeated data group, obtain relevant value change data according to the monitoring interval, wherein the relevant value change data includes the values ​​in the data items obtained before and after the monitoring interval, so as to determine the value change of each data item X in the repeated data group, and obtain the change value Bhz of each data item X between the business systems through calculation;

[0031] The comprehensive analysis unit is used to construct a data recognition model using deep learning technology, input the change value Bhz, the difference Cyd and the similarity Xsd into the data recognition model, and after dimensionless processing, fit and output the judgment index Pdzs, which is obtained by the following formula: ;

[0032] In the formula, , and are the weight values ​​of similarity Xsd, change value Bhz and difference Cyd, , and The specific value is set by the user according to the situation.

[0033] Preferably, the data processing module is used to pre-set an evaluation threshold M, and compare it with the determination index Pdzs to identify and screen out non-redundant data groups from the repeated data groups again. If the determination index Pdzs exceeds the evaluation threshold M, the corresponding data items are recorded as non-redundant data items, and statistics are performed to obtain non-redundant data groups. In combination with the repeated data groups updated in the obvious difference units, redundant data groups are determined, and the redundant data groups are compressed and stored to reduce storage space occupancy.

[0034] The industrial data analysis method based on dynamic forms includes the following steps:

[0035] S1. Use monitoring instruments to obtain monitoring data of various business systems in the industrial park, generate dynamic forms based on the monitoring data of various business systems, and define conditions for the dynamic forms to dynamically display data items X under corresponding conditions;

[0036] S2. When multiple business systems share data, the duplication between business systems is analyzed according to the data item X corresponding to the shared data to calculate the similarity Xsd, and the duplicate data group is screened based on the similarity Xsd;

[0037] S3, according to the repeated data group, identifying the characteristics of each data item X in the repeated data group in the context, and performing data context vectorization to analyze the difference in the use of each data item X in the repeated data group in different contexts to obtain the difference degree Cyd, and based on the difference degree Cyd, preliminarily screening out some non-redundant data groups from the repeated data group;

[0038] S4. Based on the data usage analysis module, relevant numerical change data is obtained according to the monitoring interval to calculate the change value Bhz, and the output judgment index Pdzs is fitted after dimensionless processing in combination with the trained data recognition model and similarity Xsd;

[0039] S5. According to the determination index Pdzs, non-redundant data groups are identified and screened out from the repeated data groups again, and the redundant data groups are compressed and stored.

[0040] The present invention provides a method and system for analyzing industrial data based on dynamic forms, which has the following beneficial effects:

[0041] (1) The system integrates monitoring data from different business systems into dynamic forms through the form construction module, and can collect and display monitoring data from various systems in real time. By defining conditions, the data items displayed in the form will be dynamically adjusted to adapt to different business needs. This enables data from multiple systems in the industrial park to be shared efficiently, reduces the existence of information islands, and improves the efficiency of collaborative work between systems. When multiple systems share data, the system calculates the similarity Xsd of data items through the repetitive analysis module and filters out duplicate data groups based on this similarity. This enables the system to effectively identify and eliminate redundant data, avoid unnecessary repeated collection and storage, thereby optimizing data storage, saving storage resources, and reducing the burden of redundant data. Through the data usage analysis module, the system can identify the characteristics of each data item in the duplicate data group in different contexts. By vectorizing the data context and analyzing the usage difference Cyd in different contexts, the system can further determine which data items have different meanings in different systems or business scenarios, thereby filtering out non-redundant data that should not be considered redundant. This analysis ensures that the system not only makes judgments based on data similarity, but also comprehensively considers the context of the data, improving the intelligence and accuracy of data processing. Through the redundancy identification module, the system not only identifies redundant data based on similarity analysis, but also judges the numerical changes in combination with the changing trend of the monitoring data, and fits the judgment index Pdzs. The system can more accurately identify redundant data groups and compress and store them. This mechanism avoids excessive data storage and ensures the simplicity and processing efficiency of the data. By comprehensively utilizing the above modules, the system can accurately identify which data items are redundant and compress and store the redundant data. This not only improves the utilization efficiency of data storage, further avoids a large amount of useless data occupying storage space, but also improves the efficiency of data processing and analysis through streamlined data sets, reducing the burden on the system. The system helps managers and decision makers in industrial parks better understand and analyze real-time monitoring data through the flexibility of dynamic forms and the intelligence of data analysis modules. By reducing redundant data and enhancing the contextual understanding of data, the analysis results provided by the system are more accurate and operational, providing stronger support for business decisions. This industrial data analysis system based on dynamic forms can greatly improve data processing efficiency and reduce the burden of redundant data storage through functions such as efficient data sharing, redundant data identification, context analysis and compressed storage. At the same time, it can provide accurate and intelligent data analysis support for industrial parks, and ultimately achieve efficient and intelligent data management and analysis.

[0042] (2) Through the preparation unit, the system can accurately determine the data items X that need to be shared according to the sharing needs of multiple business systems, and filter out relevant data from the monitoring data of each business system. This data preprocessing and aggregation function ensures the accuracy and pertinence of shared data and avoids the mixing of irrelevant data. Through the definition of conditions and rules, the system can clearly identify the shared data area, thereby optimizing the data sorting and display process and improving the efficiency of data sharing between different systems. The repetition unit effectively analyzes the data repetition between business systems by calculating the similarity Xsd of data items in the shared data area. The similarity Xsd is calculated by a formula to measure the degree of similarity between data items of two different business systems. This calculation process uses vector representation and modulus value to quantify the similarity of data items in a numerical way, so that the system can judge whether there are duplicate data items based on this value. This process not only improves the detection accuracy of duplicate data, but also avoids data redundancy caused by human factors or technical problems. By accurately screening duplicate data and performing intelligent processing, the system can maintain data consistency and interoperability among multiple business systems, ensuring the accuracy and real-time nature of shared data in different systems. This approach provides a strong guarantee for cross-system collaboration within the industrial park, avoiding data conflicts and inconsistencies caused by duplicate data. Through data optimization and standardization, the smooth exchange of data between various systems in the industrial park is promoted.

[0043] (3) The difference unit calculates the context difference Cyd based on the feature vector Y of each data item in the repeated data group, and then analyzes the differences in the use of data items in different contexts. The difference Cyd reflects the functional differences of data items in multiple systems by comparing the feature vectors of the same data items in different business systems. For example, a data item may represent the status of the equipment in one system, and may serve as a decision-making basis for production scheduling in another system. By calculating the difference, the application differences of these data items in different business systems can be accurately measured. This quantitative process of the difference enables the system to intelligently distinguish which data items have significant functional differences, thereby avoiding misjudging these data items as redundant data. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a block diagram of the industrial data analysis system based on dynamic forms of the present invention;

[0045] Figure 2 The present invention is a flow chart of the industrial data analysis method based on dynamic forms. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Example 1

[0048] See also Figure 1 , the present invention provides an industrial data analysis system based on dynamic forms, including a form construction module, a repeatability analysis module, a data usage analysis module, a redundancy identification module and a data processing module;

[0049] The form construction module is used to obtain monitoring data of each business system in the industrial park using monitoring instruments, generate dynamic forms based on the monitoring data of each business system, and define conditions for the dynamic forms to dynamically display data items X under corresponding conditions;

[0050] The repetitive analysis module is used to analyze the repetitiveness between business systems when multiple business systems share data, according to the data item X corresponding to the shared data, to calculate and obtain the similarity Xsd, and to filter the repeated data group based on the similarity Xsd;

[0051] The data usage analysis module is used to identify the features of each data item X in the repeated data group in the context according to the repeated data group, and perform data context vectorization to analyze the difference degree of usage of each data item X in the repeated data group in different contexts to obtain the difference degree Cyd, and preliminarily screen out some non-redundant data groups from the repeated data group based on the difference degree Cyd;

[0052] The redundant identification module is used to obtain relevant numerical change data based on the monitoring interval of the data usage analysis module, judge the numerical change of each data item X in the repeated data group, calculate the change value Bhz, combine the trained data recognition model and similarity Xsd, and fit the output judgment index Pdzs after dimensionless processing;

[0053] The data processing module is used to identify and filter out non-redundant data groups from repeated data groups again according to the determination index Pdzs, and compress and store the redundant data groups.

[0054] During the operation of this system, the system aggregates the monitoring data of multiple business systems and generates dynamic forms through the form construction module, which can dynamically display the corresponding data items according to actual needs. Through conditional definition and adaptive display, data from different business systems can be efficiently displayed and processed in the same form, avoiding the problem of inflexible data display caused by static form design, and improving the efficiency of data sharing and interaction. The repetitive analysis module can effectively filter out duplicate data groups by analyzing the similarity of data item X between various business systems, avoiding repeated collection and storage of the same monitoring data in multiple systems. The data usage analysis module can further determine which duplicate data is redundant and which is necessary by analyzing the data context differences, so as to accurately filter non-redundant data. This intelligent data redundancy identification can minimize the storage requirements of redundant data, save storage resources and improve storage efficiency. The data usage analysis module can identify the differences in data usage in different systems and environments by performing vectorized analysis on the features of data item X in different contexts in the duplicate data group, which provides more accurate and targeted input for subsequent data analysis and avoids data misunderstanding or erroneous analysis caused by unclear context. Through this refined analysis, the system can better support business decisions and improve the accuracy and efficiency of data analysis. Based on the data usage analysis, the redundant identification module can determine the numerical changes of repeated data items by monitoring intervals, numerical changes and model training, and output the determination index Pdzs in combination with similarity evaluation. Through this module, redundant data can not only be accurately identified, but also can be processed by compressed storage, reducing unnecessary data storage and optimizing data processing efficiency. By accurately identifying and screening out non-redundant data, the system can provide more efficient and accurate data support, helping decision makers in industrial parks to conduct real-time monitoring, adjust production strategies, optimize equipment management, etc. The resulting decision data will be more reliable and can quickly respond to changes in the production process, thereby improving the operational efficiency of the entire industrial park. The design based on dynamic forms makes the system highly scalable and flexible. In summary, the present invention significantly improves the efficiency of data sharing, storage and analysis through the intelligent construction of dynamic forms and the accurate identification of redundant data, reduces the storage requirements of redundant data, and provides strong technical support for data management and business decision-making in industrial parks.

[0055] Example 2

[0056] Please refer to Figure 1 ,Specifically: the form construction module includes a data monitoring unit and a form definition unit;

[0057] The data monitoring unit is used to provide an interface to exchange data with monitoring instruments in each business system to obtain monitoring data of each business system in the industrial park, and clean the monitoring data of each business system and convert it into a unified format (such as JSON, XML, etc.), wherein the monitoring data of each business system includes but is not limited to production status, equipment operation status, environmental parameters, etc.;

[0058] The form definition unit is used to generate a dynamic form based on the monitoring data of each business system acquired and processed by the data monitoring unit, wherein the dynamic form includes the monitoring data of each business system, and the conditions and rules of the dynamic form are defined to display the required related data items X through the defined conditions, and automatically hide the remaining data, wherein the condition and rule definitions include state conditions, user selection conditions, time and date conditions, data item enable and disable conditions, data range verification and data format verification, wherein the data item X refers to the data value collected by each sensor under the defined conditions.

[0059] Among them, the status condition determines the display of different fields based on the current status of the equipment or system (such as equipment operation status, production process status, etc.). Status conditions usually involve the start and stop of the equipment, maintenance status, etc. Example: If the production status is "pause", hide the "production progress" related data items, that is, hide the related data items other than "pause";

[0060] User selection conditions dynamically display related data items based on the user's selection in the form. When the user selects an option, the form may display specific data items or content based on the selection. Example: If the user selects the "Equipment Repair" option, the "Reason for Repair" field is displayed.

[0061] Time and date conditions: The display of certain data items is related to a specific time period or date. For example, some data is only displayed during working hours or at a specific production stage.

[0062] Data item enable and disable conditions: In addition to determining whether a data item is displayed, enable and disable conditions further control whether a data item can be edited or filled in by the user. This helps control user input permissions based on context and business needs.

[0063] Data range validation: For example, a data item of a certain numerical type (such as temperature, pressure, etc.) must be within a specific range. If it exceeds the range, an error prompt will be given.

[0064] Data format validation: Verify that the input format of the data item meets the requirements. For example, the input date must conform to a specific format, or the input temperature value must be within a reasonable range.

[0065] In this embodiment, the data monitoring unit exchanges data with the monitoring instruments in each business system by providing an interface, ensuring that data from different business systems can be smoothly integrated into the system. By cleaning and converting the data, the system can uniformly format and standardize the monitoring data from different sources such as production, equipment operation status, and environmental parameters, thereby eliminating data inconsistency and format differences, laying a solid foundation for subsequent data analysis and display. This data integration process can significantly improve data collection efficiency and further avoid errors and complexities caused by manual processing and multiple conversions. Through the form definition unit, the system dynamically generates forms based on the cleaned and formatted monitoring data. These dynamic forms can display relevant data items X according to specific needs, and automatically adjust the displayed content according to the set conditions to ensure that only data items related to the current task are displayed, which not only improves the flexibility of data display, but also simplifies the user's operation process and avoids unnecessary data interference. Through the definition of conditions and rules, the form can intelligently display, hide or enable / disable different data items, improving the user-friendliness of the system. The form definition unit supports the definition of multiple conditions and rules, such as data item display conditions, status conditions, user selection conditions, time and date conditions, etc., which enhances the configurability and adaptability of the system as much as possible.

[0066] Users can flexibly set the display mode and enabled / disabled status of data items according to different business needs to ensure the efficient operation of the form in various scenarios. The setting of these conditions and rules can not only prevent users from making mistakes during data processing, but also ensure the accuracy and compliance of the data. For example, through time and date conditions, the form only displays data within the specified time range to ensure the timeliness and accuracy of the information. The system can monitor the data entered by users in real time to ensure that it complies with the set rules through integrated data range validation, logical relationship validation, and data format validation functions. The range validation of data items can prevent data from exceeding the preset range, the logical relationship validation can ensure the logical consistency between different data items, and the data format validation can ensure the correctness of the data. These validation functions effectively prevent data entry errors and improve the accuracy and reliability of system data.

[0067] Example 3

[0068] Please refer to Figure 1 ,Specifically: the repeatability analysis module includes a preparation unit, a repeating unit and a judgment unit;

[0069] The preparation unit is used to determine the data items X involved in the sharing of the multiple business systems according to the sharing requirements of the multiple business systems when multiple business systems are sharing data, generate shared data after aggregation, delete the data items X other than the shared data from the monitoring data of each business system based on the shared data, and define the shared data area by using conditions and rules;

[0070] The repeating unit is used to analyze the repetitiveness between the business systems according to the data item X involved in the sharing of multiple business systems displayed in the shared data area, so as to calculate and obtain the similarity Xsd. The similarity Xsd is obtained by the following formula: ;

[0071] In the formula, and Vector representation of data items from two different business systems. and They are vectors and Model.

[0072] The judgment unit is used for a preset threshold, and compares the similarity Xsd with the preset threshold to screen the duplicate data group. The specific screening content is as follows:

[0073] If the similarity Xsd exceeds a preset threshold, it means and There is similarity between them, that is, when two different business systems share data, the corresponding data items in them have similarities; at this time, the corresponding data items are determined as duplicate data items and are not stored in the shared data area temporarily, and after statistics, duplicate data groups are generated;

[0074] If the similarity Xsd does not exceed the preset threshold, it means and There is no similarity between them, that is, when two different business systems share data, the corresponding data items therein do not have similarity; at this time, the data items in each business system are stored in the shared data area.

[0075] In this embodiment, when multiple business systems share data, the preparation unit can intelligently determine and summarize the data items X shared this time according to the sharing requirements of each system. By screening and processing the shared data, the system removes redundant or irrelevant data items from the monitoring data of the business system, and only retains the data items X involved in this sharing. Through the shared data area management based on conditions and rule definitions, the system can clearly define which data items are the core content of sharing, avoiding the display and storage of redundant and irrelevant data, which not only ensures the accuracy of data sharing, but also improves the efficiency of data exchange and avoids the waste of redundant data. Through the repetitive unit, the system can analyze the data items X in the shared data area, and accurately judge the data duplication between each business system based on the similarity calculation (through vector representation and modular operation). The similarity Xsd calculated by the system can reflect the similarity of the shared data items between two different business systems. If the similarity Xsd is high, it means that the data items shared by the two systems have a high similarity, and the system can effectively identify the duplicate data and avoid its storage. This analysis process can significantly reduce data redundancy, optimize data storage space, and improve the efficiency of subsequent data analysis and processing. The judgment unit can flexibly and accurately filter out duplicate data groups by comparing the similarity Xsd with the preset threshold. When the similarity exceeds the set threshold, the system determines that the data item is duplicate data and temporarily does not store it in the shared data area, thereby avoiding duplicate storage of duplicate data. Through this precise screening mechanism, the system not only effectively reduces the generation of redundant data, but also ensures the rationality and necessity of data storage. Through the collaboration of these modules, the system can reduce unnecessary data storage and sharing, and improve the data sharing efficiency between multiple business systems in the industrial park. At the same time, the efficient identification and screening of duplicate data avoids the waste of storage resources, especially in the environment of large-scale data collection and processing, which can significantly reduce storage costs. Ultimately, this mechanism will bring about data storage compression, resource savings, and improved system performance, making data analysis and decision-making more efficient and accurate.

[0076] Example 4

[0077] Please refer to Figure 1 ,Specifically: the data usage analysis module includes a vectorization unit, a difference unit and a screening unit;

[0078] The vectorization unit is used to identify, based on the repeated data group, the specific business function role of each data item X in the repeated data group in the corresponding business system, the upstream and downstream relationship of the data item, and the change and fluctuation of the data item in the time dimension, wherein the specific business function role includes sensor data, control instructions, system status and alarm signals, and is represented by classification labels; the upstream and downstream relationship of the data item refers to whether the data item X has input or output of a decision or operation, such as whether it is the input of a control signal, whether it is the basis for a production scheduling decision, and can be represented by a Boolean value to reflect whether it has an impact; the change and fluctuation of the data item in the time dimension refers to the time characteristics of the data item X, including the rate of change and the amplitude of fluctuation, which are represented as time series features; by performing data context vectorization on the specific business function role of each data item X in the repeated data group in the corresponding business system, the upstream and downstream relationship of the data item, and the change and fluctuation of the data item in the time dimension, a feature vector Y of each data item in the repeated data group is generated, and data context vectorization is the process of mapping the above features into vectors.

[0079] The difference unit is used to analyze the difference degree of use of each data item X in the repeated data group in different contexts according to the characteristic vector Y of each data item in the repeated data group, so as to obtain the difference degree Cyd, and the difference degree Cyd is obtained by the following formula: ;

[0080] Where n represents the number of data items in the repeated data group, i=1, 2, ..., n, and The feature vector of the i-th data item in the repeated data group comes from two different business systems.

[0081] The screening unit is used to pre-set a difference threshold, and compare and analyze the difference threshold with the difference degree Cyd, so as to preliminarily screen out some non-redundant data groups from the repeated data groups. The specific contents are as follows:

[0082] If the difference degree Cyd exceeds the difference threshold, it means that the context difference of the corresponding data item is in an abnormal state. At this time, the corresponding data item is recorded as a non-redundant data item and counted to obtain a non-redundant data group, which means that the uses or functions of these data items in different systems are significantly different and cannot be regarded as redundant.

[0083] If the difference degree Cyd does not exceed the difference threshold, it means that the context difference of the corresponding data item is in a normal state. At this time, the corresponding data item is recorded as a redundant data item, which means that the use or function of these data items in different systems is not significantly different and can be regarded as redundant.

[0084] For contextual analysis of data, how to identify the characteristics of data items in different contexts and how to measure their differences are the key to judging redundant data and non-redundant data. Contextual analysis focuses more on the functional role, purpose, related information and usage scenarios of data in different systems. Therefore, the core of contextual analysis is to understand the business context of each data item and quantify its differences in some way, and finally determine whether the data items have the same purpose in different systems, and then determine whether they are redundant.

[0085] In this embodiment, the vectorization unit converts these complex context information into feature vectors by analyzing the specific business function roles, upstream and downstream relationships, and changes and fluctuations in the time dimension of each data item in the repeated data group. Through this data context vectorization process, the system can not only identify the actual role of each data item in the business (such as sensor data, control instructions, system status, etc.), but also reveal the change pattern of the data item in different time intervals. This vectorization process enables the business context and time characteristics of the data item to be accurately quantified, providing a solid foundation for subsequent difference analysis. Through the difference unit, the system analyzes the feature vector Y of each data item in the repeated data group, calculates and obtains the difference Cyd. The difference Cyd quantifies the degree of difference in the use of the same data item in the two business systems in the context, helping the system to identify which data items have significant functional differences in different contexts. This analysis process not only reveals the potential differences between the repeated data items, but also clearly reflects which data items perform different roles in different business systems, and therefore cannot be regarded as redundant. The calculation and analysis of the difference Cyd can ensure whether the data items between different systems have actual use differences, thereby helping to accurately determine which data items belong to non-redundant data. The screening unit can screen out non-redundant data groups according to the difference degree Cyd by comparing with the pre-set difference threshold. When the difference degree Cyd exceeds the set threshold, the system recognizes that the function or use of the data item in different systems is significantly different, indicating that the data item is non-redundant and must be retained. This screening mechanism effectively reduces the storage of redundant data, so that only data items with unique value and function are stored in the shared area, thereby optimizing data storage and improving data processing efficiency. By vectorizing the context information of the data item and combining it with difference analysis, the system can deeply understand the role and changes of each data item in different systems. Compared with traditional data analysis methods, this analysis is not only limited to direct comparison of values ​​and attributes, but also takes into account the business functions, upstream and downstream relationships and time characteristics of the data items, so it has higher intelligence and accuracy. This analysis capability enables the system to process complex multi-source data and provide more scientific and intelligent decision-making basis for data cleaning, storage and sharing. By accurately screening non-redundant data, the system can effectively avoid unnecessary redundant data storage and transmission, reduce system resource consumption, and improve the response speed and accuracy of data analysis. At the same time, this also reduces the redundant information interference in subsequent data processing, making the data more concise and clear in practical applications, thereby optimizing the overall efficiency of industrial data analysis.

[0086] Example 5

[0087] Please refer to Figure 1 ,Specifically: the redundant identification module includes an obvious difference unit and a comprehensive analysis unit;

[0088] The obvious difference unit is used to re-update the repeated data group according to the screening unit, and on the basis of the re-updated repeated data group, obtain relevant value change data according to the monitoring interval, wherein the relevant value change data includes the values ​​in the data items obtained before and after the monitoring interval, so as to determine the value change of each data item X in the repeated data group, and obtain the change value Bhz of each data item X between the business systems through calculation;

[0089] The comprehensive analysis unit is used to construct a data recognition model using deep learning technology, input the change value Bhz, the difference Cyd and the similarity Xsd into the data recognition model, and after dimensionless processing, fit and output the judgment index Pdzs, which is obtained by the following formula: ;

[0090] In the formula, , and are the weight values ​​of similarity Xsd, change value Bhz and difference Cyd, respectively, where 0< <1,0< <1,0< <1, , and The specific value is set by the user according to the situation.

[0091] The data processing module is used to pre-set an evaluation threshold M, and compare it with the determination index Pdzs to identify and screen out non-redundant data groups from the repeated data groups again. If the determination index Pdzs exceeds the evaluation threshold M, the corresponding data items are recorded as non-redundant data items, and statistics are performed to obtain non-redundant data groups. In combination with the repeated data groups updated again in the obvious difference units, redundant data groups are determined, and the redundant data groups are compressed and stored to reduce the storage space occupied.

[0092] Among them, the setting method of the evaluation threshold M is as follows: according to the data sharing data between other business systems, similarly, the judgment index Pdzs is obtained, and the average value and standard deviation of the judgment index Pdzs are obtained in combination with the statistical algorithm. Based on the average value and standard deviation of the judgment index Pdzs, the evaluation threshold M is set: Evaluation threshold M = average value of judgment index Pdzs + k * standard deviation of judgment index Pdzs, wherein k is a constant, usually taking a value of 1-3, corresponding to different confidence levels, and the specific value is set by the user (according to actual conditions).

[0093] In this embodiment, the obvious difference unit calculates the change value Bhz based on the re-updated repeated data group by obtaining the relevant numerical change data before and after the monitoring interval, which can accurately capture the change of the data item in the time dimension. This process can not only identify the fluctuation of the data item at different time points, but also compare and analyze the change trend of the same data item in different business systems. This method can effectively eliminate those false redundant data caused by time differences or external factors, and provide strong guarantees for the accurate storage and subsequent processing of data. At the same time, the change value Bhz can be used as a basis for measuring the importance of data, further optimize the data screening process, and reduce unnecessary data storage. The comprehensive analysis unit takes the change value Bhz, the difference Cyd and the similarity Xsd as input, performs dimensionless processing through the constructed deep learning data recognition model, and calculates the judgment index Pdzs. This process can comprehensively consider the change of data items from different sources, functional differences and similarities, and perform intelligent evaluation and screening. By adjusting the weight value, the user can customize the sensitivity of the model according to specific business needs, thereby optimizing the effect of data analysis. The output of the model, namely the judgment index Pdzs, can provide an accurate basis for further identifying non-redundant data groups. On this basis, the redundant data groups can be compressed and stored to reduce the occupancy rate of storage space. The data processing module further screens out non-redundant data groups by setting the evaluation threshold M and combining the judgment index Pdzs, and compresses and stores the redundant data. This mechanism significantly reduces the occupancy of storage resources by redundant data, optimizes the data storage and management process, and thus improves the storage efficiency and data processing capabilities of the entire system. At the same time, this redundant data identification and compression storage processing can effectively reduce storage costs and computing overhead, and provide a more efficient storage environment for subsequent data analysis. Through deep learning technology and dimensionless processing, the system can quickly adapt to changing data environments and needs, and provide efficient redundant data identification and compression storage services. This flexibility and efficiency enable the system to be widely used in different industrial data analysis scenarios, improving the application scope and scalability of the system.

[0094] Example 6

[0095] Please refer to Figure 2 Specifically: the industrial data analysis method based on dynamic forms includes the following steps:

[0096] S1. Use monitoring instruments to obtain monitoring data of various business systems in the industrial park, generate dynamic forms based on the monitoring data of various business systems, and define conditions for the dynamic forms to dynamically display data items X under corresponding conditions;

[0097] S2. When multiple business systems share data, the duplication between business systems is analyzed according to the data item X corresponding to the shared data to calculate the similarity Xsd, and the duplicate data group is screened based on the similarity Xsd;

[0098] S3, according to the repeated data group, identifying the characteristics of each data item X in the repeated data group in the context, and performing data context vectorization to analyze the difference in the use of each data item X in the repeated data group in different contexts to obtain the difference degree Cyd, and based on the difference degree Cyd, preliminarily screening out some non-redundant data groups from the repeated data group;

[0099] S4. Based on the data usage analysis module, relevant numerical change data is obtained according to the monitoring interval to calculate the change value Bhz, and the output judgment index Pdzs is fitted after dimensionless processing in combination with the trained data recognition model and similarity Xsd;

[0100] S5. According to the determination index Pdzs, non-redundant data groups are identified and screened out from the repeated data groups again, and the redundant data groups are compressed and stored.

[0101] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Industrial data analysis system based on dynamic forms, characterized by: It includes form construction module, repetitive analysis module, data usage analysis module, redundancy identification module and data processing module; The form construction module is used to obtain monitoring data of each business system in the industrial park using monitoring instruments, generate dynamic forms based on the monitoring data of each business system, and define conditions for the dynamic forms to dynamically display data items X under corresponding conditions; The repetitive analysis module is used to analyze the repetitiveness between business systems when multiple business systems share data, according to the data item X corresponding to the shared data, to calculate and obtain the similarity Xsd, and to filter the repeated data group based on the similarity Xsd; The data usage analysis module is used to identify the features of each data item X in the repeated data group in the context according to the repeated data group, and perform data context vectorization to analyze the difference degree of usage of each data item X in the repeated data group in different contexts to obtain the difference degree Cyd, and preliminarily screen out some non-redundant data groups from the repeated data group based on the difference degree Cyd; The redundant identification module is used to obtain relevant numerical change data based on the monitoring interval on the basis of the data usage analysis module to calculate the change value Bhz, and combine the trained data recognition model and similarity Xsd, and after dimensionless processing, fit the output judgment index Pdzs; The data processing module is used to identify and filter out non-redundant data groups from repeated data groups again according to the determination index Pdzs, and compress and store the redundant data groups.

2. The industrial data analysis system based on dynamic forms according to claim 1 is characterized in that: The form construction module includes a data monitoring unit and a form definition unit; The data monitoring unit is used to provide an interface to exchange data with monitoring instruments in each business system to obtain monitoring data of each business system in the industrial park, and to clean the monitoring data of each business system and convert it into a unified format; The form definition unit is used to generate a dynamic form based on the monitoring data of each business system acquired and processed by the data monitoring unit, wherein the dynamic form includes the monitoring data of each business system, and the conditions and rules of the dynamic form are defined to display the required related data items X through the defined conditions, and automatically hide the remaining data, wherein the condition and rule definitions include state conditions, user selection conditions, time and date conditions, data item enable and disable conditions, data range verification and data format verification, wherein the data item X refers to the data value collected by each sensor under the defined conditions.

3. The industrial data analysis system based on dynamic forms according to claim 2 is characterized in that: The repeatability analysis module includes a preparation unit, a repeating unit and a judgment unit; The preparation unit is used to determine the data items X involved in the sharing of the multiple business systems according to the sharing requirements of the multiple business systems when multiple business systems are sharing data, generate shared data after aggregation, delete the data items X other than the shared data from the monitoring data of each business system based on the shared data, and define the shared data area by using conditions and rules; The repeating unit is used to analyze the repetitiveness between the business systems according to the data item X involved in the sharing of multiple business systems displayed in the shared data area, so as to calculate and obtain the similarity Xsd. The similarity Xsd is obtained by the following formula: ； In the formula, and Vector representation of data items from two different business systems, and They are vectors and Model.

4. The industrial data analysis system based on dynamic forms according to claim 3 is characterized in that: The judgment unit is used for a preset threshold, and compares the similarity Xsd with the preset threshold to screen the duplicate data group. The specific screening content is as follows: If the similarity Xsd exceeds a preset threshold, it means and If there is similarity between them, the corresponding data item is determined as a duplicate data item and is not stored in the shared data area temporarily. After statistics, a duplicate data group is generated; If the similarity Xsd does not exceed the preset threshold, it means and There is no similarity between them yet, so the data items in each business system are stored in the shared data area.

5. The industrial data analysis system based on dynamic forms according to claim 4 is characterized in that: The data usage analysis module includes a vectorization unit, a difference unit and a screening unit; The vectorization unit is used to identify, based on the repeated data group, the specific business function role of each data item X in the repeated data group in the corresponding business system, the upstream and downstream relationship of the data item, and the change and fluctuation of the data item in the time dimension, wherein the specific business function role includes sensor data, control instructions, system status and alarm signals, and is represented by classification labels; the upstream and downstream relationship of the data item refers to whether the data item X has input or output of a decision or operation; the change and fluctuation of the data item in the time dimension refers to the time characteristics of the data item X, including the rate of change and the amplitude of fluctuation, which are represented as time series features; the specific business function role of each data item X in the repeated data group in the corresponding business system, the upstream and downstream relationship of the data item, and the change and fluctuation of the data item in the time dimension are vectorized to generate a feature vector Y of each data item in the repeated data group.

6. The industrial data analysis system based on dynamic forms according to claim 5 is characterized in that: The difference unit is used to analyze the difference degree of use of each data item X in the repeated data group in different contexts according to the characteristic vector Y of each data item in the repeated data group, so as to obtain the difference degree Cyd, and the difference degree Cyd is obtained by the following formula: ; Where n represents the number of data items in the repeated data group, i=1, 2, ..., n, and The feature vector of the i-th data item in the repeated data group comes from two different business systems.

7. The industrial data analysis system based on dynamic forms according to claim 5 is characterized in that: The screening unit is used to pre-set a difference threshold, and compare and analyze the difference threshold with the difference degree Cyd, so as to preliminarily screen out some non-redundant data groups from the repeated data groups. The specific contents are as follows: If the difference degree Cyd exceeds the difference threshold, it means that the context difference of the corresponding data item is in an abnormal state. At this time, the corresponding data item is recorded as a non-redundant data item and counted to obtain a non-redundant data group; If the difference degree Cyd does not exceed the difference threshold, it means that the context difference of the corresponding data item is in a normal state, and the corresponding data item is recorded as a redundant data item.

8. The industrial data analysis system based on dynamic forms according to claim 7 is characterized in that: The redundancy identification module includes a significant difference unit and a comprehensive analysis unit; The obvious difference unit is used to re-update the repeated data group according to the screening unit, and on the basis of the re-updated repeated data group, obtain relevant value change data according to the monitoring interval, wherein the relevant value change data includes the values ​​in the data items obtained before and after the monitoring interval, so as to determine the value change of each data item X in the repeated data group, and obtain the change value Bhz of each data item X between the business systems through calculation; The comprehensive analysis unit is used to construct a data recognition model using deep learning technology, input the change value Bhz, the difference Cyd and the similarity Xsd into the data recognition model, and after dimensionless processing, fit and output the judgment index Pdzs, which is obtained by the following formula: ； In the formula, , and are the weight values ​​of similarity Xsd, change value Bhz and difference Cyd, , and The specific value is set by the user according to the situation.

9. The industrial data analysis system based on dynamic forms according to claim 8 is characterized in that: The data processing module is used to pre-set an evaluation threshold M, and compare it with the determination index Pdzs to identify and screen out non-redundant data groups from the repeated data groups again. If the determination index Pdzs exceeds the evaluation threshold M, the corresponding data items are recorded as non-redundant data items, and statistics are performed to obtain non-redundant data groups. In combination with the repeated data groups updated again in the obvious difference units, redundant data groups are determined, and the redundant data groups are compressed and stored to reduce the storage space occupied.

10. A method for analyzing industrial data based on dynamic forms, used to implement a system for analyzing industrial data based on dynamic forms as described in any one of claims 1 to 9, characterized in that: The following steps are involved: S1. Use monitoring instruments to obtain monitoring data of various business systems in the industrial park, generate dynamic forms based on the monitoring data of various business systems, and define conditions for the dynamic forms to dynamically display data items X under corresponding conditions; S2. When multiple business systems share data, the duplication between business systems is analyzed according to the data item X corresponding to the shared data to calculate the similarity Xsd, and the duplicate data group is screened based on the similarity Xsd; S3, according to the repeated data group, identifying the characteristics of each data item X in the repeated data group in the context, and performing data context vectorization to analyze the difference in the use of each data item X in the repeated data group in different contexts to obtain the difference degree Cyd, and based on the difference degree Cyd, preliminarily screening out some non-redundant data groups from the repeated data group; S4. Based on the data usage analysis module, relevant numerical change data is obtained according to the monitoring interval to calculate the change value Bhz, and the output judgment index Pdzs is fitted after dimensionless processing in combination with the trained data recognition model and similarity Xsd; S5. According to the determination index Pdzs, non-redundant data groups are identified and screened out from the repeated data groups again, and the redundant data groups are compressed and stored.

Images