Industrial internet data processing system and safety management method thereof
By building an adaptive data quality evaluation model and using historical data to continuously learn and adjust, the problem of inaccurate data quality evaluation in the existing technology is solved, priority determination and efficient repair of data repair are achieved, and adaptability and production efficiency of data processing systems are improved.
Patent Information
- Application Number
- CN202411356054.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-02-25
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
It is difficult for the existing technology to adjust the evaluation standards and methods in a timely manner according to the dynamic changes in data in the industrial Internet environment, resulting in inaccurate data quality assessment and ineffective determination of the data that needs to be repaired and its priorities, which affects the efficiency and effectiveness of data repair.
By building an adaptive data quality evaluation model, we use historical data to continuously learn and adjust, adapt to different industrial environments and data characteristics, and use recurrent neural networks and optimization algorithms to evaluate data quality and fix priority.
It realizes dynamic adjustment of evaluation standards and methods based on data characteristics, improves the accuracy and adaptability of data quality evaluation, and prioritizes the repair of data that has a greater impact on production, improves the efficiency of data repair, and reduces the impact of data errors on production.
Smart Images
Figure CN119515127A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to an industrial Internet data processing system and a security management method thereof. Background Art
[0002] With the rapid development of information technology, the industrial Internet is gradually becoming a key force in promoting the transformation of industrial intelligence. A large amount of data is generated in the industrial production process, including real-time data and historical data. These data are crucial for optimizing production processes, improving production efficiency, and ensuring production safety. The current industrial Internet data processing faces many challenges. The data collected by sensors often have problems such as noise and missing values. Effective preprocessing is required to ensure the accuracy and reliability of the data. The evaluation and repair of data quality is a complex process. Traditional data quality evaluation methods are often difficult to adapt to the dynamic and diversity of data in the industrial Internet environment, and cannot accurately determine the data that needs to be repaired and its priority. In the industrial Internet environment, data security management is also a key issue. Since industrial production involves important production equipment and critical infrastructure, data leakage, tampering or loss may lead to serious production accidents and economic losses. It is necessary to monitor the data quality evaluation and repair process and the security management process in real time to ensure the security and integrity of the data. In order to solve these problems in industrial Internet data processing, an innovative industrial Internet data processing system and its security management method are urgently needed.
[0003] However, the current common solutions have many shortcomings, including: the existing data quality assessment models are usually fixed, and the assessment standards and methods cannot be adjusted in time according to the dynamic changes of data in the industrial Internet environment. The data characteristics in the industrial production process may change with time, production conditions and other factors, which makes it difficult for existing models to accurately assess data quality; the existing technology often does not have clear priority classification standards when determining the data that needs to be repaired. This will lead to the inability to prioritize the repair of data problems that have the greatest impact on production when resources are limited, reducing the efficiency and effectiveness of data repair. Summary of the invention
[0004] In view of the fact that there is often no clear priority classification standard when determining the data that needs to be repaired, this may lead to problems such as failure to give priority to repairing the data that has the greatest impact on production when resources are limited. The present invention is proposed.
[0005] Therefore, the problem to be solved by the present invention is how to provide a method for timely adjusting evaluation standards and methods according to the dynamic changes of data in the industrial Internet environment to improve the quality of evaluation data.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] In the first aspect, an embodiment of the present invention provides an industrial Internet data processing security management method, which includes collecting real-time data and historical data from industrial production sites through sensors and preprocessing them; building an adaptive data quality assessment model based on the historical data and evaluating the real-time data; determining the data that needs to be repaired and its priority based on the data quality assessment results, and repairing the data that needs to be repaired in order of priority; and monitoring the data quality assessment and repair process and the security management process in real time.
[0008] As a preferred solution of the industrial Internet data processing security management method described in the present invention, the real-time data includes: equipment operation status data, production process data and environmental data; the historical data includes: production process data, environmental data, equipment maintenance data and energy consumption data; the evaluation indicators include accuracy, completeness, consistency and timeliness; the priorities include: the importance of the data, the degree of impact on the production process and the severity of data quality issues.
[0009] As a preferred solution of the industrial Internet data processing security management method described in the present invention, the steps of constructing an adaptive data quality assessment model are: clarifying the business needs and data quality requirements of the industrial Internet data processing system; determining the index system for data quality assessment based on the business needs and data quality requirements; constructing an adaptive data quality assessment model based on a recurrent neural network RNN and selecting an optimization algorithm; selecting a suitable assessment method based on the assessment indicators of data characteristics; training the assessment model through historical data; applying the trained assessment model to the industrial Internet data processing system to detect and evaluate real-time data; the specific formula for data quality assessment is as follows:
[0010]
[0011] Among them, F(d) is the data quality evaluation function; B is the quantitative value of business requirements; Q is the data quality evaluation value; O is the quantitative value of the recurrent neural network output; M is the quantitative value of the evaluation method; T is the quantitative value of the model after historical data training; E is the quantitative value of real-time data detection evaluation.
[0012] As a preferred solution of the industrial Internet data processing security management method of the present invention, the specific formula of the optimization algorithm is as follows:
[0013]
[0014] Among them, O pis the optimization algorithm; B is the business demand quantification value; Q is the data quality assessment value; O is the quantification value of the recurrent neural network output value; R is the data repair demand value; D i f is the data difference measurement value; d i It is a collection of data in the industrial Internet data processing system.
[0015] As a preferred solution of the industrial Internet data processing security management method described in the present invention, the step of determining the priority order includes the following steps: classifying the collected data according to data type and source; conducting a preliminary quality assessment on each type of data, determining the key factors affecting the data priority, and setting corresponding weights for each key factor; constructing a priority evaluation function to calculate the priority score of each data item; based on the priority score results, sorting the data items from high to low priority, determining the data that needs to be repaired and its priority order; and adjusting the weight value in time according to changes in priority, as well as new demands and new problems in the production process.
[0016] As a preferred solution of the industrial Internet data processing security management method of the present invention, the specific formula of the priority evaluation function is as follows:
[0017]
[0018] Among them, P r is the priority score; Q is the data quality assessment value; R is the data repair requirement value; and P is the data priority determination value.
[0019] As a preferred solution of the industrial Internet data processing security management method of the present invention, the specific formula for adjusting the weight value is as follows:
[0020]
[0021] Among them, ω is the adjusted weight value; ω0 is the initial key weight function; P r is the priority score; D(d i ) is the data detection change value; m is the total number of key factors.
[0022] In the second aspect, in order to further solve the above-mentioned technical problems, the embodiments of the present invention provide an industrial Internet data processing system, which includes: a data acquisition module, which is used to collect real-time data and historical data of industrial production sites; a model building module, which is used to build an adaptive data quality assessment model based on the historical data and evaluate the real-time data; a data repair module, which is used to determine the data that needs to be repaired and the priority according to the data quality assessment results, and repair the data that needs to be repaired according to the priority order; a security management module, which is used to monitor the data quality assessment and repair process and the security management process in real time.
[0023] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of an industrial Internet data processing security management method as described in the first aspect of the present invention.
[0024] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored, wherein: when the computer program is executed by a processor, it implements any step of an industrial Internet data processing security management method as described in the first aspect of the present invention.
[0025] The beneficial effects of the present invention are as follows: the present invention can continuously learn and adjust according to historical data by utilizing an adaptive data quality assessment model, adapt to different industrial environments and data characteristics, so that the system has stronger adaptability and can operate stably in various complex industrial scenarios; by determining the priority of data repair, data that has a greater impact on the production process can be repaired first, thereby improving the efficiency of data repair, helping to reduce the impact of data errors on production and reducing production losses. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0027] Figure 1 This is a flow chart for implementing the present invention in Example 1;
[0028] Figure 2 This is a comparison chart of data evaluation and repair of the present invention in Example 2. DETAILED DESCRIPTION
[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0031] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0032] Example 1
[0033] Reference Figure 1 , which is the first embodiment of the present invention, and provides an industrial Internet data processing security management method, comprising the following steps:
[0034] S1: Collect real-time data and historical data from industrial production sites through sensors and perform preprocessing.
[0035] Preferably, Figure 1 The figure shows the implementation process of the present invention. First, real-time data and historical data are collected from the industrial production site through sensors and preprocessed. Then, an adaptive data quality assessment model is constructed based on the historical data and the real-time data is evaluated. According to the data quality assessment results, the data that needs to be repaired and their priority are determined. The data that needs to be repaired is repaired according to the priority order. The data quality assessment and repair process as well as the security management process are monitored in real time.
[0036] Furthermore, the real-time data includes: equipment operation status data, production process data and environmental data.
[0037] Furthermore, the historical data includes: production process data, environmental data, equipment maintenance data and energy consumption data.
[0038] Specifically, preprocessing ensures the quality and consistency of data by preprocessing real-time data and historical data in real time on production nodes, including data cleaning, noise reduction, and standardization.
[0039] S2: Build an adaptive data quality assessment model based on historical data and evaluate real-time data.
[0040] Preferably, evaluation indicators include accuracy, completeness, consistency and timeliness.
[0041] Specifically, the steps to build an adaptive data quality assessment model are:
[0042] Clarify the business needs and data quality requirements of the industrial Internet data processing system;
[0043] Determine the indicator system for data quality assessment based on business needs and data quality requirements;
[0044] Build an adaptive data quality assessment model based on recurrent neural network (RNN) and select an optimization algorithm;
[0045] Select appropriate evaluation methods based on the evaluation indicators of data characteristics;
[0046] Train the evaluation model using historical data;
[0047] Apply the trained evaluation model to the industrial Internet data processing system to detect and evaluate real-time data;
[0048] Furthermore, the specific formula for data quality assessment is as follows:
[0049]
[0050] Among them, F(d) is the data quality evaluation function; B is the business demand quantification value; Q is the data quality evaluation value; O is the recurrent neural network output quantification value; M is the evaluation method quantification value; T is the model quantification value after historical data training; E is the real-time data detection evaluation quantification value;
[0051] Furthermore, the specific formula of the optimization algorithm is as follows:
[0052]
[0053] Among them, O p is the optimization algorithm; B is the business demand quantification value; Q is the data quality assessment value; O is the quantification value of the recurrent neural network output value; R is the data repair demand value; D i f is the data difference measurement value; d i It is a collection of data in the industrial Internet data processing system.
[0054] Furthermore, an adaptive data quality assessment model is constructed, which can continuously learn and adjust the assessment criteria based on historical data, thereby improving the accuracy and adaptability of data quality assessment. By evaluating real-time data, data quality problems can be discovered in a timely manner, providing a basis for data repair.
[0055] S3: According to the data quality assessment results, determine the data that needs to be repaired and its priority, and repair the data that needs to be repaired in order of priority.
[0056] Preferably, the priorities include: the importance of the data, the degree of impact on the production process, and the severity of the data quality problem.
[0057] Specifically, the steps for determining the priority order include the following steps:
[0058] Classify the collected data according to data type and source;
[0059] Conduct a preliminary quality assessment on each type of data, determine the key factors that affect data priority, and set corresponding weights for each key factor;
[0060] Construct a priority evaluation function to calculate the priority score of each data item;
[0061] According to the priority score results, the data items are sorted from high to low priority to determine the data that needs to be repaired and its priority order;
[0062] Adjust the weight value in time according to the changes in priorities, new demands and new problems in the production process.
[0063] Furthermore, the specific formula of the priority evaluation function is as follows:
[0064]
[0065] Among them, P r is the priority score; Q is the data quality assessment value; R is the data repair requirement value; and P is the data priority determination value.
[0066] Furthermore, the specific formula for adjusting the weight value is as follows:
[0067]
[0068] Among them, ω is the adjusted weight value; ω0 is the initial key weight function; P r is the priority score; D(d i ) is the data detection change value; m is the total number of key factors.
[0069] Preferably, the data that needs to be repaired and its priority are determined based on the data quality assessment results, so that the data that has a greater impact on the production process can be repaired first, thereby improving the efficiency and results of data repair. Repairing by priority order can ensure that important data is processed in a timely manner, reducing the impact of data quality issues on the production process.
[0070] S4: Real-time monitoring of data quality assessment and repair process and security management process.
[0071] Specifically, by real-time monitoring of the data quality assessment and repair process and the security management process, problems can be discovered in a timely manner and corresponding measures can be taken, thereby improving the security and stability of the system, understanding the progress and effects of data processing in real time, and providing timely information support for decision-making.
[0072] This embodiment also provides an industrial Internet data processing system, including: a data acquisition module, used to collect real-time data and historical data of industrial production sites; a model construction module, used to build an adaptive data quality assessment model based on historical data and evaluate real-time data; a data repair module, used to determine the data that needs to be repaired and the priority according to the data quality assessment results, and repair the data that needs to be repaired according to the priority order; a security management module, used to monitor the data quality assessment and repair process and the security management process in real time.
[0073] This embodiment also provides a computer device, which is suitable for an industrial Internet data processing security management method, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement an industrial Internet data processing security management method as proposed in the above embodiment.
[0074] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.
[0075] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, an industrial Internet data processing security management method as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM for short), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM for short), programmable read-only memory (Programmable Red-Only Memory, PROM for short), read-only memory (Read-Only Memory, ROM for short), magnetic storage, flash memory, disk or optical disk.
[0076] In summary, the present invention utilizes an adaptive data quality assessment model to continuously learn and adjust according to historical data, adapt to different industrial environments and data characteristics, so that the system has stronger adaptability and can operate stably in various complex industrial scenarios; by determining the priority of data repair, data that has a greater impact on the production process can be repaired first, thereby improving the efficiency of data repair, helping to reduce the impact of data errors on production and reducing production losses.
[0077] Example 2
[0078] Referring to Tables 1 to 3, the second embodiment of the present invention is shown. This embodiment is different from the first embodiment in that, in order to verify its beneficial effects, operating data and related instructions of the present invention in an actual environment are provided.
[0079] Table 1 shows the comparison of data collection items performed by the traditional method and the method of the present invention in this example.
[0080] Table 1 Data collection comparison table
[0081]
[0082] Table 2 shows the comparison of data evaluation and repair between the traditional method and the method of the present invention in this example.
[0083] Table 2 Comparison of data evaluation and repair
[0084] Data processing stage Traditional methods Method of the present invention Data preprocessing 30％ 80％ Data quality assessment 70％ 90％ Data repair priority determination 40％ 80％ Data repair effect 50％ 85％
[0085] Table 3 shows the comparison of real-time monitoring between the traditional method and the method of the present invention in this example.
[0086] Table 3 Real-time monitoring comparison table
[0087] Compare Projects Traditional methods Method of the present invention Degree of monitoring Limited monitoring Comprehensive monitoring Early warning mechanism none Yes, issue an alarm in time Problem response time Longer Short, fast processing Monitoring visibility Low High, intuitive display of data changes Monitoring data analysis depth shallow Digging deep into potential problems
[0088] It can be seen from the above table that the present invention can give priority to repairing data that has a greater impact on the production process, thereby improving the efficiency of data repair.
[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for industrial Internet data processing security management, characterized by: include: Collect real-time and historical data from industrial production sites through sensors and pre-process them; Building an adaptive data quality assessment model based on the historical data and evaluating the real-time data; According to the data quality assessment results, determine the data that needs to be repaired and its priority, and repair the data that needs to be repaired according to the priority order; Through real-time monitoring of data quality assessment and repair process and security management process.
2. The industrial Internet data processing security management method according to claim 1, characterized in that: The real-time data includes: equipment operation status data, production process data and environmental data; The historical data includes: production process data, environmental data, equipment maintenance data and energy consumption data; The evaluation criteria include accuracy, completeness, consistency and timeliness; The priorities include: the importance of the data, the impact on the production process and the severity of the data quality issues.
3. The industrial Internet data processing security management method according to claim 2, characterized in that: The steps of constructing the adaptive data quality assessment model are: Clarify the business needs and data quality requirements of the industrial Internet data processing system; Determine the indicator system for data quality assessment based on the business needs and data quality requirements; Build an adaptive data quality assessment model based on recurrent neural network (RNN) and select an optimization algorithm; Select appropriate evaluation methods based on the evaluation indicators of data characteristics; Train the adaptive data quality assessment model using historical data; Apply the trained evaluation model to the industrial Internet data processing system to detect and evaluate real-time data; The specific formula for data quality assessment is as follows: Among them, F(d) is the data quality evaluation function; B is the business demand quantification value; Q is the data quality evaluation value; O is the recurrent neural network output quantification value; M is the evaluation method quantification value; T is the quantized value of the model after historical data training; E is the quantized value of real-time data detection and evaluation.
4. The industrial Internet data processing security management method according to claim 3, characterized in that: The specific formula of the optimization algorithm is as follows: Among them, O p is the optimization algorithm; B is the business demand quantification value; Q is the data quality assessment value; O is the quantification value of the recurrent neural network output value; R is the data repair demand value; D i f is the data difference measurement value; d i It is a collection of data in the industrial Internet data processing system.
5. The industrial Internet data processing security management method according to claim 4, characterized in that: The step of determining the priority order comprises the following steps: Classify the collected data according to data type and source; Conduct a preliminary quality assessment on each type of data, determine the key factors that affect data priority, and set corresponding weights for each key factor; Construct a priority evaluation function to calculate the priority score of each data item; According to the priority score results, the data items are sorted from high to low priority to determine the data that needs to be repaired and its priority order; Adjust the weight value in time according to the changes in priorities, new demands and new problems in the production process.
6. The industrial Internet data processing security management method according to claim 5, characterized in that: The specific formula of the priority evaluation function is as follows: Among them, P r is the priority score; Q is the data quality assessment value; R is the data repair requirement value; and P is the data priority determination value.
7. The industrial Internet data processing security management method according to claim 6, characterized in that: The specific formula for adjusting the weight value is as follows: Among them, ω is the adjusted weight value; ω0 is the initial key weight function; P r is the priority score; D(d i ) is the data detection change value; m is the total number of key factors.
8. An industrial Internet data processing system, based on an industrial Internet data processing security management method according to any one of claims 1 to 7, characterized in that: include, Data acquisition module, used to collect real-time data and historical data from industrial production sites; A model building module, used to build an adaptive data quality assessment model based on the historical data and evaluate the real-time data; The data repair module is used to determine the data that needs to be repaired and the priority according to the data quality assessment results, and repair the data that needs to be repaired according to the priority order; The security management module is used to monitor the data quality assessment and repair process and the security management process in real time.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of an industrial Internet data processing security management method described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of an industrial Internet data processing security management method as described in any one of claims 1 to 7 are implemented.