Basic data quality safety management and control system and method based on data acquisition

Abstract

The invention relates to the technical field of data collection, and discloses a basic data quality safety management and control system and method based on data collection, and the system comprises a data collection monitoring module which collects an inverter operation data set and an inverter state data set for numbering, the data acquisition and evaluation module calculates an inverter state index based on the inverter state data set and judges the safety of the inverter operation data set and the inverter state data set, and the data acquisition and storage module calculates the data receiving integrity based on the received inverter operation data set and the inverter state data set and judges the safety of the inverter operation data set and the inverter state data set. And judging whether the data is complete or not based on a data receiving integrity calculation result, sending the data to the platform library unit for storage under the condition that the data is judged to be complete, and implementing a data exception strategy under the condition that the data is judged to be incomplete. According to the method, unsafe data is intercepted from the source, database pollution is prevented, and data integrity and reliability entering a platform library unit are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data acquisition, in particular to a basic data quality safety management and control system and method based on data acquisition. BACKGROUND

[0002] Under the strategic guidance of "carbon peak and carbon neutrality", power data runs through all links of the entire power generation system like a bond. At present, China is accelerating the green and low-carbon transformation process of urban and rural construction, among which the household distributed photovoltaic industry in the field of new rural construction is developing rapidly. The power system, as a large and complex energy supply system, covers power generation, power transmission, power transformation, power distribution and power utilization. Its core mission is to efficiently convert primary energy in nature into electric energy and ensure that these electric energy can be safely and stably delivered to millions of households and various industrial users, which is called the "energy lifeline" supporting the normal operation of modern society.

[0003] Among many key devices, the inverter plays a crucial role. With the increasing proportion of new energy power generation methods such as photovoltaic and wind power in the energy structure, the importance of inverters is increasingly prominent. On the one hand, it effectively solves the "electricity form adaptation" problem faced by new energy power generation - as photovoltaic panels, energy storage devices and other outputs are direct current, while traditional power grids operate on alternating current, inverters use efficient direct current to alternating current conversion technology to enable new energy generated electricity to be smoothly connected to the grid. On the other hand, advanced inverters also have intelligent active adjustment capabilities, which can sense the voltage and frequency fluctuations of the power grid in real time and adjust the output power to assist in maintaining the stability of the power grid. For example, when detecting that the voltage of the power grid is low, the inverter will timely increase the output voltage to effectively alleviate the voltage drop phenomenon, thereby effectively ensuring the safe and stable operation and reliable power supply of the entire power system.

[0004] Current inverter data transmission is difficult to guarantee the security and integrity of data during transmission in the security protection layer, lacks effective transmission protection mechanism, and causes abnormal or unsafe data to enter the database, thereby affecting the credibility of data application. SUMMARY

[0005] In view of the shortcomings of the prior art, the present application provides a basic data quality safety management and control system and method based on data acquisition, which has the advantages of monitoring and quantitatively evaluating the multi-dimensional parameters of the inverter state data set, ensuring the safety and effectiveness of the inverter operation data set and its own collection from the source, avoiding subjective judgment bias, intercepting unsafe data from the source, preventing pollution of the database, ensuring that the data entering the platform library unit are complete and reliable through the block mechanism, maintaining the accuracy and stability of the data storage system, laying a foundation for long-term data retention and reuse, etc.

[0006] To achieve the above object, the present application provides the following technical solutions: a basic data quality safety management and control system based on data collection, comprising a data collection monitoring module, a data collection evaluation module and a data collection storage module; The data collection monitoring module comprises a data collection unit and a data monitoring unit, the data collection unit is used for collecting inverter operation data sets and numbering the collected inverter operation data sets, and the data monitoring unit is used for monitoring inverter state data sets in the data collection process of the inverter operation data sets and numbering the inverter state data sets; The data collection evaluation module comprises a data calculation unit and a data judgment unit, the data calculation unit is used for calculating an inverter state index based on the inverter state data sets, and the data judgment unit is used for judging the safety of the inverter operation data sets and the inverter state data sets based on the inverter state index, and sending the inverter operation data sets and the inverter state data sets to the data collection storage module in the case of judging safety, and sending a data collection exception alarm signal in the case of judging insecurity; The data collection storage module comprises a data review unit and a platform library unit, the data review unit is used for calculating data reception integrity based on the received inverter operation data sets and inverter state data sets, and judging whether the data is complete based on the data reception integrity calculation result, and sending the data to the platform library unit for saving in the case of judging that the data is complete, and implementing a data exception strategy in the case of judging that the data is incomplete.

[0007] Preferably, the inverter operation data sets comprise AC output voltage, AC output current, AC output power, DC input voltage, DC input current, power factor, power generation, cumulative operation time length, output load rate and power grid frequency, each parameter in the inverter operation data sets is numbered in turn to form a numbering expression of the inverter operation data sets, and the numbering expression of the inverter operation data sets is: .

[0008] Preferably, the inverter state data sets comprise inverter operation mode, equipment fault code, core component temperature, energy storage battery state, key collection module state, data collection frequency, transmission pass rate and communication signal strength, each parameter in the inverter state data sets is numbered in turn to form a numbering expression of the inverter state data sets, and the numbering expression of the inverter state data sets is: .

[0009] Preferably, the inverter state index The calculation formula is:

[0010] In the calculation formula, represents a penalty factor of inverter operation mode, represents a penalty factor of equipment fault code; represents a core component temperature state score, represents a storage battery state score, represents a key acquisition module state score, represents a data acquisition frequency score, represents a transmission pass rate score, represents a communication signal strength score; 、 、 、 、 、 、 all represent weights, and, .

[0011] Preferably, when the calculation result of the inverter state index is greater than or equal to 60, the data judgment unit determines that the current inverter operation data set and inverter state data set are safe, and sends the inverter operation data set and inverter state data set to the data acquisition storage module; when the calculation result of the inverter state index is less than 60, the data judgment unit determines that the current inverter operation data set and inverter state data set are not safe, and sends a data acquisition exception alarm signal.

[0012] Preferably, the data judgment unit sends the inverter operation data set and inverter state data set to the data acquisition storage module after being divided into blocks, and the division expression is:

[0013] In the expression, represents the first block, represents the block, and simultaneously represents the number of blocks, the upper right superscript represents an inverter operation data set block, and the upper right superscript represents an inverter state data set block; Under each block, the data size of each inverter operation data block and each inverter state data block is equal.

[0014] Preferably, the data auditing unit calculates the data reception integrity based on the received blocks of the inverter operation data set and inverter state data set, and the calculation formula of the data reception integrity is:

[0015] represents the initial data size of the inverter operation data set, represents the initial data size of the inverter operation data set, represents the sum of the sizes of the received inverter operation data blocks, represents the sum of the sizes of the received inverter operation data blocks, represents the data reception integrity of the inverter operation data set. represents the initial data size of the inverter state data set, represents the initial data size of the inverter state data set, represents the sum of the sizes of the received inverter state data blocks, represents the sum of the sizes of the received inverter state data blocks,

[0016] Preferably, when the calculation result of the data reception integrity is less than 2, the data auditing unit determines that the data is incomplete, and when the calculation result of the data reception integrity is equal to 2, it is determined that the data is complete.

[0017] Preferably, the implementation of the data exception strategy is specifically: When the data auditing unit determines that the data is incomplete, the size of each initial data block in the data judging unit is compared with the size of the received data block in the data auditing unit, and a report is generated for feedback after sorting according to the difference in data comparison results.

[0018] A basic data quality safety management and control method based on data collection is applied to a basic data quality safety management and control system based on data collection, comprising the following steps: Step one, collect the inverter operation data set and the inverter state data set, and number and save them; Step two, calculate the inverter state index based on the inverter state data set, and determine the safety of the inverter operation data set and the inverter state data set; Step three, save the data in blocks under the condition of determining safety, and calculate the data reception integrity, and send a data collection exception alarm signal under the condition of determining insecurity; Step four, determine whether the data is complete based on the calculation result of the data reception integrity, save the data under the condition of determining that the data is complete, and implement a data exception strategy under the condition of determining that the data is incomplete.

[0019] Compared with the prior art, the basic data quality safety management and control system and method based on data collection have the following beneficial effects:​​ 1、The present application guarantees the safety and effectiveness of the inverter operation data set and the safety of the inverter itself by monitoring and quantitatively evaluating the multi-dimensional parameters of the inverter state data set, and based on these parameters, the inverter state index that comprehensively reflects the overall state of the inverter is calculated by combining the penalty factor and the normalized score and assigning reasonable weights to each parameter, avoiding the one-sidedness of single index evaluation, converting the abstract device state into quantifiable numerical value, and objectively reflecting the safety of data collection. The monitoring and judgment mechanism based on the state data set not only avoids subjective judgment deviation, intercepts unsafe data from the source, prevents database pollution, but also realizes early warning of faults, guarantees continuous and stable data collection process, and provides effective data support for subsequent energy efficiency analysis, capacity planning and other applications.

[0020] 2、The present application guarantees the integrity of the inverter operation data set and the inverter state data set by data block mechanism, after judging the safety of the data, the two types of data are divided into n equal size blocks according to the block expression, realizing the decomposition of large data set into small units, which not only reduces the complexity of large-scale data transmission, avoids loss or damage caused by too large data volume in single transmission, but also provides convenience for subsequent data audit and storage. Through the running data integrity calculation formula, the receiving condition of each type of data is accurately checked, and when the data is incomplete, the system generates a report feedback by comparing the size and sorting difference of the initial block and the received block, which is convenient for quickly positioning the missing or damaged block, making the data integrity check more efficient and accurate, avoiding missing local problems due to overall data check, and at the same time, block storage also facilitates subsequent data calling, ensuring that the data entering the platform library unit are complete and reliable, maintaining the accuracy and stability of the data storage system, and laying a foundation for long-term data retention and reuse. BRIEF DESCRIPTION OF DRAWINGS

[0021] Fig. 1 The present application is a system flowchart; Fig. 2 The present application is a method step diagram. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0023] Please refer to Figs. 1-2 A basic data quality safety control system based on data collection, comprising a data collection monitoring module, a data collection evaluation module and a data collection storage module. The data acquisition monitoring module comprises a data acquisition unit and a data monitoring unit, the data acquisition unit is configured to acquire an inverter operation data set and number the acquired inverter operation data set; The inverter operation data set comprises AC output voltage, AC output current, AC output power, DC input voltage, DC input current, power factor, power generation, cumulative operation time length, output load rate and power grid frequency, and each parameter in the inverter operation data set is sequentially numbered to form a number expression of the inverter operation data set, and the number expression of the inverter operation data set is: ; The inverter operation data set covers multiple parameters such as AC output voltage, AC output current and AC output power, and is a basic carrier for the system to obtain core information of the inverter operation, directly reflects the power generation capacity and power grid adaptability of the inverter, and forms an ordered set through numbering, not only enables the data acquisition unit to accurately locate the collection source and attribute of each parameter, avoids parameter confusion, but also provides a clear identification basis for subsequent data tracing, when it is necessary to check the power generation data at a certain time, the corresponding key parameters such as AC output power and power grid frequency can be quickly matched through the number expression, and the data tracing efficiency is greatly improved; The data monitoring unit is configured to monitor an inverter state data set in a data acquisition process of the inverter operation data set and number the inverter state data set; The inverter state data set comprises inverter operation mode, equipment fault code, core component temperature, energy storage battery state, key acquisition module state, data acquisition frequency, transmission pass rate and communication signal strength, each parameter in the inverter state data set is sequentially numbered to form a number expression of the inverter state data set, and the number expression of the inverter state data set is: ; The inverter state data set comprises multiple parameters such as inverter operation mode and equipment fault code, and constitutes a core judgment basis for judging data quality and safety in the control system, and realizes real-time control of the stability of the acquisition process, and abnormalities in the acquisition process can be found in time; The data acquisition evaluation module comprises a data calculation unit and a data judgment unit, the data calculation unit is configured to calculate an inverter state index based on the inverter state data set; The calculation formula of the inverter state index is:

[0024] In the calculation formula, represents an inverter operation mode penalty factor, represents an equipment fault code penalty factor; represents a core component temperature state score, representing the energy storage battery state score, representing the key acquisition module state score, representing the data acquisition frequency score, representing the transmission pass rate score, representing the communication signal strength score; 、 、 、 、 、 、 all represent weights, and, ; The inverter operation mode penalty factor gives a specific score based on the inverter operation mode, and is 1 when the operation mode is an ideal state; is 0.6 when in standby or debugging mode; is 0.3 when in shutdown or shutdown mode; and is 0 when in fault mode; The device fault code penalty factor gives a score based on the device fault code level, and is 1 when there is no fault; is 0.8 when it is a warning; is 0.5 when it is a general fault, and is 0 when it is a serious fault; The core component temperature state score, the energy storage battery state score, the key acquisition module state score, the data acquisition frequency score, the transmission pass rate score, and the communication signal strength score are all calculated using a normalization method, which is the current existing technology, and will not be described again. The inverter state index considers multiple dimensions of factors such as inverter operation mode, device fault code, core component temperature, energy storage battery state, key acquisition module, data acquisition frequency, transmission pass rate, and communication signal strength. By quantitatively scoring these different aspects and assigning appropriate weights, the overall operation of the inverter can be comprehensively and objectively reflected, avoiding the one-sidedness of single index evaluation, making the originally abstract device state quantifiable and comparable, which helps to more intuitively understand the health status of the inverter and the safety of data acquisition, and timely discovers potential problems, provides data support for formulating reasonable maintenance strategies, at the same time, can discover abnormal trends before the device appears obvious faults, realize early warning of faults, improve the accuracy and efficiency of fault diagnosis, reduce downtime and maintenance cost; The data judgment unit determines the safety of the inverter operation data set and the inverter state data set based on the inverter state index. When the calculation result of the inverter state index is greater than or equal to 60, the data judgment unit determines that the current inverter operation data set and inverter state data set are safe, and sends the inverter operation data set and inverter state data set to the data acquisition storage module; When the inverter state index If the calculation result is less than 60, the data judgment unit determines that the current inverter operation data set and inverter state data set are unsafe, and sends a data collection exception alarm signal; Through the explicit judgment rule, the judgment deviation caused by subjective judgment difference is avoided, the safe data and unsafe data can be accurately distinguished, the unsafe data is intercepted from the source, the database is prevented from being polluted by the unsafe data in the storage link, and the high quality of the system storage data is ensured. The data judgment unit sends the inverter operation data set and inverter state data set to the data collection storage module after being divided into blocks, and the block expression is:

[0025] In the expression, represents the first block, represents the second block, and simultaneously represents the number of blocks, the upper right superscript represents the inverter operation data set block, and the upper right superscript represents the inverter state data set block. Under each block, the data size of each inverter operation data block and each inverter state data block is equal. The data collection storage module includes a data auditing unit and a platform library unit. The data auditing unit calculates the data reception integrity based on the received inverter operation data set and inverter state data set, determines whether the data is complete based on the data reception integrity calculation result, sends the data to the platform library unit for storage in the case of determining that the data is complete, and implements a data exception strategy in the case of determining that the data is not complete. The inverter operation data set and the state data set are divided into blocks, so that the huge data is divided into relatively small units. Whether the data is transmitted, stored or analyzed, each small block can be operated more efficiently, and the complexity and performance problems caused by processing the whole large-scale data are avoided. The data auditing unit calculates the data reception integrity based on the blocks of the received inverter operation data set and inverter state data set. The data reception integrity The calculation formula is:

[0026] In the calculation formula, represents the initial data size of the inverter operation data set, represents the sum of the sizes of the inverter operation data blocks in the received inverter operation data set, represents the reception data integrity of the inverter operation data set. representing the initial data size of the inverter state data set, representing the data size of the received inverter state data set the sum of the sizes of the inverter state data blocks, representing the data integrity of the received inverter state data set; when the data integrity is less than 2, the data auditing unit determines that the data is incomplete, and when the data integrity is equal to 2, it is determined that the data is complete; The specific implementation of the data exception strategy is: When the data auditing unit determines that the data is incomplete, the data extraction unit compares the size of each initial data block with the size of the received data block in the data auditing unit, and generates a report according to the difference in the data comparison result, and feeds back the report; The data auditing unit calculates the data integrity based on the received data set, which can timely find out whether the data is lost, damaged, etc. during transmission or collection, and ensure that the data entering the storage link is complete and reliable, providing quality assurance for subsequent analysis and application based on these data. The platform library unit performs different operations according to the data integrity determination result, saves the complete data for subsequent call and analysis, and implements the exception strategy for the incomplete data, which helps to maintain the accuracy and stability of the entire data storage system, and ensures that the stored data is usable and valuable; The specific method of the above-mentioned data collection-based basic data quality safety management and control system is as follows, including the following steps: Step one, collect the inverter operation data set and the inverter state data set, and number and save them; Step two, calculate the inverter state index based on the inverter state data set, and determine the safety of the inverter operation data set and the inverter state data set; Step three, save the data in blocks under the condition of determining safety, and calculate the data integrity, and send a data collection exception alarm signal under the condition of determining insecurity; Step four, determine whether the data is complete based on the data integrity calculation result, save the data under the condition of determining that the data is complete, and implement the data exception strategy under the condition of determining that the data is incomplete.

[0027] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A basic data quality and security management system based on data acquisition, characterized in that, It includes a data acquisition and monitoring module, a data acquisition and evaluation module, and a data acquisition and storage module; The data acquisition and monitoring module includes a data acquisition unit and a data monitoring unit. The data acquisition unit is used to acquire inverter operation datasets and number the acquired inverter operation datasets. The data monitoring unit is used to monitor inverter status datasets during the data acquisition process of inverter operation datasets and number the inverter status datasets. The data acquisition and evaluation module includes a data calculation unit and a data judgment unit. The data calculation unit is used to calculate the inverter status index based on the inverter status dataset. The data judgment unit determines the safety of the inverter operation dataset and the inverter status dataset based on the inverter status index. If the data is deemed safe, the inverter operation dataset and the inverter status dataset are sent to the data acquisition and storage module. If the data is deemed unsafe, a data acquisition anomaly alarm signal is sent. The data acquisition and storage module includes a data review unit and a platform library unit. The data review unit calculates the data reception integrity based on the received inverter operation dataset and inverter status dataset, and determines whether the data is complete based on the data reception integrity calculation result. If the data is complete, the data is sent to the platform library unit for storage. If the data is incomplete, a data anomaly strategy is implemented.

2. The basic data quality and security management system based on data acquisition according to claim 1, characterized in that: The inverter operation dataset includes AC output voltage, AC output current, AC output power, DC input voltage, DC input current, power factor, power generation, cumulative operating time, output load rate, and grid frequency. Each parameter in the inverter operation dataset is sequentially numbered to form a numbered expression for the inverter operation dataset. The numbered expression for the inverter operation dataset is as follows: .

3. The basic data quality and security management system based on data acquisition according to claim 2, characterized in that: The inverter status dataset includes inverter operating mode, equipment fault codes, core component temperature, energy storage battery status, key acquisition module status, data acquisition frequency, transmission throughput, and communication signal strength. Each parameter in the inverter status dataset is sequentially numbered to form a numbered expression for the inverter status dataset. The numbered expression for the inverter status dataset is as follows: .

4. The basic data quality and security management system based on data acquisition according to claim 3, characterized in that: The inverter state index The calculation formula is: In the calculation formula, Represents the inverter operating mode penalty factor. Represents the penalty factor for equipment fault codes; This represents the temperature condition score of the core components. Represents the state rating of energy storage batteries. This represents the status score of the key data acquisition module. Represents the score for data collection frequency. Represents the transmission throughput score. Represents the communication signal strength score; , , , , , , Both represent weights, and, .

5. A basic data quality and security management system based on data acquisition according to claim 4, characterized in that: When the inverter state index If the calculation result is greater than or equal to 60, the data judgment unit determines that the current inverter operation dataset and inverter status dataset are safe, and sends the inverter operation dataset and inverter status dataset to the data acquisition and storage module. When the inverter state index If the calculation result is less than 60, the data judgment unit determines that the current inverter operation dataset and inverter status dataset are unsafe and sends a data acquisition abnormality alarm signal.

6. The basic data quality and security management system based on data acquisition according to claim 5, characterized in that: The data judgment unit divides the inverter operation dataset and the inverter status dataset into blocks and sends them to the data acquisition and storage module. The block expression is as follows: In the expression, Represents the first block. Representing the Each block, Also represents the number of blocks, indicated by the upper right index. Represents the inverter running dataset block, top right index Represents a block of inverter state data; Within each block, the data size of each inverter operation data block and each inverter status data block is equal.

7. A basic data quality and security management system based on data acquisition according to claim 6, characterized in that: The data verification unit calculates data reception integrity based on blocks of the received inverter operation dataset and inverter status dataset. The calculation formula is: In the calculation formula, This represents the initial data size of the inverter's running dataset. The received inverter running data is concentrated. The sum of the data block sizes of each inverter operation This represents the integrity of the received data in the inverter's operating dataset; This represents the initial data size of the inverter state dataset. The data represents the received inverter state data. The sum of the sizes of the inverter status data blocks This represents the integrity of the received data in the inverter status dataset.

8. A basic data quality and security management system based on data acquisition according to claim 7, characterized in that: When the data reception is complete If the calculation result is less than 2, the data review unit determines that the data is incomplete. If the data reception is complete... If the calculation result is equal to 2, the data is considered complete.

9. A basic data quality and security management system based on data acquisition according to claim 8, characterized in that: The specific implementation of the data anomaly strategy is as follows: When the data review unit determines that the data is incomplete, it extracts the size of each initial data block in the data judgment unit and compares it with the size of the data block received in the data review unit. Then, it sorts the data according to the difference in the comparison results and generates a report for feedback.

10. A method for basic data quality and security management based on data acquisition, based on the basic data quality and security management system described in any one of claims 1 to 9, characterized in that, Includes the following steps: Collect inverter operation datasets and inverter status datasets, and save them with numbers; The inverter state index is calculated based on the inverter state dataset to determine the safety of the inverter operation dataset and the inverter state dataset. If the data is deemed safe, it is stored in blocks, and the integrity of the received data is calculated. If the data is deemed unsafe, an alarm signal for abnormal data acquisition is sent. The integrity of the received data is determined based on the data integrity calculation results. If the data is determined to be complete, it is saved; if the data is determined to be incomplete, a data anomaly policy is implemented.

Images