Production power data quality monitoring and visualization method and system

Abstract

The invention discloses a production power data quality monitoring and visualization method and system, and relates to the technical field of power data monitoring, and the method comprises the steps: collecting first production power data uploaded by each target enterprise; preprocessing the first production power data, and taking the preprocessed data as second production power data; obtaining a data quality dynamic monitoring analysis model according to the target analysis process; performing quality monitoring on the second production power data based on the data quality dynamic monitoring analysis model, and obtaining a data quality monitoring result; and visually displaying the data quality monitoring result. The method comprises the steps of preprocessing first production power data to obtain second production power data; and the second production power data is monitored based on the data quality dynamic monitoring and analysis model to obtain a data quality monitoring result, and the data quality monitoring result is visualized, automatically uploaded and monitored, so that the subsequent data analysis precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power data monitoring, and in particular to a method and system for monitoring and visualizing production power data quality. Background Art

[0002] At present, accidents caused by illegal and irregular production behaviors in coal mines occur frequently. In order to accurately judge illegal and irregular production behaviors in coal mines, electricity consumption data of coal mining enterprises can be collected for power big data analysis. The method of the prior art includes: obtaining a first device object; monitoring and sampling the first device object to obtain a first device indicator parameter; using the principle of data twin technology to construct a first twin model of the first device object, and marking it to the first twin model; establishing a target power monitoring twin model of the target power grid; analyzing the target parameter set of the target power monitoring twin model to generate target prediction information; and pre-regulating the target power grid. The above invention solves the technical problem in the prior art that the comprehensiveness and efficiency of power monitoring in the power monitoring system are low due to the low degree of comprehensive and refined monitoring of each node in the power monitoring system, and achieves the goal of improving the comprehensive and refined monitoring degree of each node in the power monitoring system, thereby achieving the effect of improving the comprehensiveness and efficiency of power monitoring in the power monitoring system.

[0003] However, when accessing multiple types of safety production related data, there are many sources of analysis data and a complex network environment. When the type and quality of data cannot meet the needs of power big data analysis applications, the accuracy of subsequent model construction is low. In view of this, a production power data quality monitoring and visualization system and method is urgently needed to at least solve the above shortcomings. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method and system for monitoring and visualizing production power data quality to solve the problem of numerous analysis data sources and complex network environment when accessing multiple types of production safety related data.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for monitoring and visualizing production power data quality, comprising: collecting first production power data uploaded by each target enterprise; preprocessing the first production power data, and using the preprocessed data as second production power data; obtaining a data quality dynamic monitoring and analysis model according to a target analysis process; based on the data quality dynamic monitoring and analysis model, performing quality monitoring on the second production power data, and obtaining data quality monitoring results; and visually displaying the data quality monitoring results.

[0008] As a preferred solution of the production power data quality monitoring and visualization method of the present invention, the collecting of the first production power data uploaded by each target enterprise includes:

[0009] The first production power data uploaded by each target enterprise is collected through a data sharing platform, wherein the data sharing platform is based on a big data distributed network storage architecture.

[0010] As a preferred solution of the method for monitoring and visualizing the quality of power production data described in the present invention, preprocessing the first power production data includes:

[0011] Preprocessing the first production power data based on multiple preprocessing steps, wherein the preprocessing steps include data cleaning, data integration, data reduction and data conversion;

[0012] The data cleaning is to detect inconsistencies, identify noise data, filter and correct the first production power data; the data integration is to integrate data from multiple data sources to form a centralized and unified database; the data summarization is to reduce the size of the data set without compromising the accuracy of the analysis results, including dimensionality reduction, data reduction, and data sampling; the data conversion is to process the first production power data based on rules or metadata, and based on models and learning to achieve data unification.

[0013] As a preferred solution of the method for monitoring and visualizing the quality of production power data described in the present invention, the dynamic monitoring and analysis model for data quality obtained according to the target analysis process includes:

[0014] Obtain historical data review records;

[0015] Determining the target analysis process of the first reviewer according to the historical data review record;

[0016] The target analysis process is to determine the purpose of the first reviewer's review based on the historical data review records, and to determine and analyze the review process in which the purpose of the review is data quality comparison, that is, the first reviewer analyzes the transmission status, access data volume, number of accessed coal mining enterprises, data qualification rate and enterprise data stability based on the historical production power data and the production power data at the time of review;

[0017] According to the target analysis process, a data quality dynamic monitoring and analysis model is trained.

[0018] As a preferred solution of the method for monitoring and visualizing the quality of production power data described in the present invention, wherein: according to the target analysis process, the training data quality dynamic monitoring and analysis model includes:

[0019] The target analysis process obtains a first analysis statement;

[0020] According to the first analysis speech, determining whether the speech time of the first analysis speech meets the analysis screen extraction condition;

[0021] If the analysis screen extraction condition is met, the analysis screen and the second analysis speech are extracted;

[0022] Determining analysis content items according to the sight line features of the analysis picture;

[0023] Based on the second analysis speech, the analysis content item and the preset analysis model, a data quality dynamic monitoring analysis model is trained.

[0024] As a preferred solution of the method for monitoring and visualizing the quality of production power data described in the present invention, the target analysis process further includes:

[0025] Obtaining the scheduling authority of the first reviewer;

[0026] If the scheduling authority of the first reviewer includes the review of historical data review records, the corresponding first reviewer will be used as the second reviewer;

[0027] Acquire the preceding operation information before the time at which the second reviewer reviews the corresponding historical data review record;

[0028] Predicting the review process according to the preceding operation information, obtaining the review process prediction result, and using it as the review purpose of the second reviewing party;

[0029] If the scheduling authority of the first reviewer does not include the review of historical data review records, the corresponding first reviewer will be regarded as the third reviewer;

[0030] A request form for the third reviewer to review the corresponding historical data review record is obtained, and the request form is parsed to obtain the review purpose of the third reviewer.

[0031] As a preferred solution of the method for monitoring and visualizing the quality of production power data described in the present invention, the visual display of the data quality monitoring result includes:

[0032] Display abnormal monitoring results on a visual dashboard;

[0033] Performing a necessity analysis of video tracing according to the abnormal type of the abnormal monitoring result;

[0034] If it is necessary to perform a video tracing necessity analysis, the second production power data corresponding to the abnormal monitoring result is used as the third production power data, and the equipment target corresponding to the third production power data is determined, and the video data of the equipment target is obtained;

[0035] Temporally aligning the third production power data with the video data of the device target, and generating a retrospective progress;

[0036] Determine a progress mark on the tracing progress, perform progress tracing back based on a preset tracing back speed, and obtain facial features of an observer during the tracing back process;

[0037] Determine the feature matching result of the features according to the facial features of the observer and the preset progress mark in the backtracking process, generate the backtracking result to be confirmed and the confirmation instruction of the backtracking result to be confirmed;

[0038] Obtaining instruction reply information of the confirmation instruction according to the confirmation instruction of the backtracking result to be confirmed;

[0039] According to the instruction reply information, a target backtracking result is determined, and the target backtracking result is displayed on a visual dashboard.

[0040] In a second aspect, the present invention provides a production power data quality monitoring and visualization system, comprising:

[0041] A data collection module, used to collect the first production power data uploaded by each target enterprise;

[0042] a preprocessing module, configured to preprocess the first production power data and use the preprocessed data as second production power data;

[0043] The first acquisition module is used to acquire a data quality dynamic monitoring and analysis model according to a target analysis process;

[0044] a second acquisition module, configured to perform quality monitoring on the second production power data based on the data quality dynamic monitoring and analysis model, and obtain a data quality monitoring result;

[0045] The visualization display module is used to visualize the data quality monitoring results.

[0046] In a third aspect, the present invention provides a computing device, comprising:

[0047] Memory and processor;

[0048] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method are implemented.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the method when executed by a processor.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a method and system for monitoring and visualizing the quality of production power data, which pre-processes the first production power data uploaded by each target enterprise to obtain the second production power data; introduces a data quality dynamic monitoring and analysis model to perform quality monitoring on the second production power data, obtains the data quality monitoring results and displays them visually, thereby realizing the automatic uploading and automatic quality monitoring of production power data and improving the accuracy of subsequent data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0052] Figure 1 This is an overall flow chart of a method for monitoring and visualizing production power data quality according to an embodiment of the present invention;

[0053] Figure 2 A flowchart of a system for monitoring and visualizing production power data quality according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of a first acquisition module of a production power data quality monitoring and visualization system according to an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a visualization display module of a production power data quality monitoring and visualization method system according to an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of a data analysis system for a method and system for monitoring and visualizing production power data quality according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0058] Example 1

[0059] Reference Figure 1 According to an embodiment of the present invention, a method for monitoring and visualizing production power data quality is provided, which specifically includes the following steps:

[0060] S101: Collecting first production power data uploaded by each target enterprise;

[0061] In the embodiment of the present application, the first production power data uploaded by each target enterprise is collected through a data sharing platform, wherein the data sharing platform is based on a big data distributed network storage architecture, the target enterprise is a coal industry enterprise, and the first production power data is data uploaded by the target enterprise during the power production process, including: enterprise power consumption, basic information, safety monitoring and personnel positioning;

[0062] It should be noted that the data sharing platform in the present invention adopts a big data distributed network storage architecture, which stores a large number of data elements in a dispersed manner on multiple nodes according to the type of data, transmission frequency, etc., thereby improving the scalability of the metadata management architecture, estimating the number of nodes required based on the amount of data and the workload of calculation, and dynamically migrating data between nodes to achieve load balancing; at the same time, when a node fails, the data can be restored through a copy mechanism, which has less impact on upper-level applications.

[0063] S102: preprocessing the first production power data, and using the preprocessed data as the second production power data;

[0064] In an embodiment of the present application, the first production power data is preprocessed based on multiple preprocessing links, wherein the preprocessing links include data cleaning, data integration, data reduction and data conversion;

[0065] In a feasible implementation manner, data cleaning is to detect inconsistencies in the first production power data, identify noise data, filter and correct data, which can improve the consistency, accuracy, authenticity and availability of the data; data integration is to integrate data from multiple data sources to form a centralized and unified database, which is conducive to improving the integrity, consistency, security and availability of big data; data induction is to reduce the size of the data set without compromising the accuracy of the analysis results, including dimension reduction, data reduction, and data sampling, to increase the value density of big data, that is, to increase the value of big data storage; data conversion is to process the first production power data based on rules or metadata, and based on models and learning, to achieve data unification and improve the consistency and availability of big data.

[0066] Furthermore, the second production power data is pre-processed data, which has been free of errors and anomalies and has been standardized or normalized, making it more suitable for subsequent analysis;

[0067] It should be noted that with the continuous advancement of the production process of coal mining enterprises, the data timeliness is high, the data differences between enterprises are large, the data quality is uneven, and the platform data collection itself has many defects. In addition, the data is easily affected by noise data, missing data values ​​and data conflicts during the push, transmission and reception process. Therefore, the present invention processes the collected first production power data based on multiple preprocessing links to obtain the second production power data, which greatly improves the data quality and the accuracy of subsequent analysis.

[0068] S103: Acquire a data quality dynamic monitoring and analysis model according to the target analysis process;

[0069] In the embodiment of the present application, the process of obtaining the data quality dynamic monitoring and analysis model is as follows:

[0070] Obtaining historical data retrieval records, where historical data retrieval records are records of manual retrieval of historical power production data, including the time of retrieval, the identity of the retrieval person, the type of data retrieval and the quantity;

[0071] According to the historical data access records, determine the target analysis process of the first access party, where the target analysis process is to determine the access purpose of the first access party according to the historical data access records, determine and analyze the access process for data quality comparison, that is, the first access party analyzes the transmission status, access data volume, number of accessed coal mining enterprises, data qualification rate and enterprise data stability based on the historical production power data and the production power data at the time of access;

[0072] According to the target analysis process, a data quality dynamic monitoring and analysis model is trained. When the data quality dynamic monitoring and analysis model is trained according to the target analysis process, a neural network model is used to learn the target analysis process to obtain an AI model that assists manual dynamic monitoring of data quality. The learning process of the neural network model is: the target analysis process is used as training data for model training until the model training reaches a convergence state.

[0073] It should be noted that the present embodiment introduces historical data review records, determines the target analysis process of the first reviewer, and trains a dynamic monitoring and analysis model for data quality according to the target analysis process, which is more reasonable.

[0074] Furthermore, according to the target analysis process, the training data quality dynamic monitoring and analysis model includes:

[0075] Analyze the target analysis process and obtain the first analysis speech, wherein the first analysis speech is the conversation information of the personnel during the analysis process;

[0076] According to the first analysis speech, it is determined whether the speech time of the first analysis speech meets the analysis screen extraction condition, wherein the analysis screen extraction condition is a determination condition of whether to extract the dashboard interface being analyzed by the corresponding personnel, for example, if the first analysis speech contains the preset visualization dashboard viewing semantics, the extraction of the analysis screen is triggered;

[0077] If the analysis screen extraction condition is met, the analysis screen and the second analysis speech are extracted, wherein the second analysis speech is the first analysis speech after the speech time, and the speech time interval between speech items of the second analysis speech is less than a preset speech time interval threshold (for example: 4 seconds);

[0078] Determine the analysis content items according to the sight line features of the analysis screen, wherein the sight line features are the sight line landing point and the stay time of the analysis screen, and the analysis content items are the data items analyzed on the analysis screen;

[0079] Based on the second analysis speech, analysis content items and the preset analysis model, a data quality dynamic monitoring analysis model is trained.

[0080] Further, based on the second analysis speech, the analysis content item and the preset analysis model, the training data quality dynamic monitoring analysis model includes:

[0081] Expanding the analysis content item and the analysis speech item of the second analysis speech on the time axis; wherein when expanding, the analysis content item and the analysis speech item correspond to the time points of the corresponding analysis / corresponding speech on the time axis one by one;

[0082] Traversing each analysis content item in turn, and taking the analysis content item being traversed as the target analysis content item;

[0083] Determine a first time point of a target analysis content item on the timeline and a second time point of an analysis content item next to the target analysis content item on the timeline;

[0084] Obtaining analysis speech items between a first time point and a second time point on the time axis and using them as speech items to be matched;

[0085] Obtaining the associated semantic items of the target analysis content item; wherein the associated semantic items are: semantics related to the analysis of the target analysis content item, which are preset manually, for example: if the target analysis content item is: the stability of the production power data uploaded by enterprise A, then the associated semantic items are: the description semantics of enterprise A, the upload cycle semantics of the production power data uploaded, etc.;

[0086] Match the speech item to be matched with the associated semantic items in sequence, and if there is a match, determine the target analysis speech item, and associate the target analysis speech item with the target analysis content item;

[0087] After all analysis content items are traversed, the data quality dynamic monitoring analysis model is trained based on the target analysis content items and the corresponding target analysis speech items and according to the preset AI analysis model.

[0088] It should be noted that the present invention parses the target analysis process to obtain the first analysis speech of the manual analysis process, introduces the analysis screen extraction condition, and determines whether the speech moment of the first analysis speech meets the analysis screen extraction condition; when the analysis screen extraction condition is met, the analysis screen and the second analysis speech corresponding to the speech moment are extracted, and the speech time interval between the speech items of the second analysis speech is less than the preset speech time interval threshold, indicating that it is continuous speech content within a period of time; the analysis content item is determined according to the analysis screen, and the analysis speech items of the analysis content item and the second analysis speech are both unfolded on the time axis, and it is determined that the target analysis content item being traversed and the analysis content item to be discussed next under the target analysis content item correspond to the speech item to be matched on the time axis, and the speech corresponding to the speech item to be matched is issued immediately after the recognition moment of the target analysis content item, and is more relevant; in addition, the preset associated semantic item of the target analysis content item is introduced, the speech item to be matched and the associated semantic item are matched, and the semantically matched target analysis speech item is determined. The target analysis speech items correspond to the target analysis content items, which are more targeted. The preset AI analysis model (neural network model) is introduced to specifically learn the target analysis speech items and the corresponding target analysis content items, learn the analysis logic, and obtain the data quality dynamic monitoring analysis model, which greatly improves the training efficiency of the model.

[0089] Furthermore, the target analysis process also includes:

[0090] Obtain the scheduling authority of the first reviewer;

[0091] If the scheduling authority of the first reviewer includes the review of historical data review records, the corresponding first reviewer is used as the second reviewer; the preceding operation information before the review time of the second reviewer reviewing the corresponding historical data review records is obtained; the review process is predicted based on the preceding operation information, and the review process prediction result is obtained and used as the review purpose of the second reviewer;

[0092] If the scheduling authority of the first reviewer does not include the review of historical data review records, the corresponding first reviewer will be regarded as the third reviewer; a request form for the third reviewer to review the corresponding historical data review records is obtained, and the request form is parsed to obtain the review purpose of the third reviewer.

[0093] It should be noted that the present invention introduces the scheduling authority of the first reviewer. Generally, when the first reviewer views the historical data review record, the platform will authenticate its identity. If the authority of the first reviewer is high enough, the first reviewer will directly review it. At this time, the system cannot directly obtain the purpose of the second reviewer's review, so the previous operation information is introduced for determination; if the authority of the first reviewer is not enough, considering the situation of temporary viewing, a request form is introduced. After the third reviewer fills in the request form, the purpose of the review can be determined by parsing. The process of determining the purpose of the review is more appropriate.

[0094] S104: Based on the data quality dynamic monitoring and analysis model, quality monitoring is performed on the second production power data, and a data quality monitoring result is obtained;

[0095] In an embodiment of the present application, the quality monitoring of the second production power data based on the data quality dynamic monitoring and analysis model is as follows: the stability and integrity of the access data are monitored in real time according to the data quality dynamic monitoring and analysis model, and the monitoring and analysis results are displayed through a visualization page, and the data quality monitoring results (current transmission status, access data volume, number of accessed coal mining enterprises, data qualification rate and enterprise data stability analysis) are dynamically viewed.

[0096] S105: Visually displaying the data quality monitoring results;

[0097] The specific process is:

[0098] The abnormal monitoring results are displayed on a visual dashboard, where the visual dashboard is a graphical interface that displays production power data and monitoring results;

[0099] The necessity analysis of video tracing is conducted based on the abnormal type of abnormal monitoring results, where the abnormal type refers to the type of abnormality, such as unstable data upload, missing data, etc. The necessity analysis of video tracing is: whether it is necessary to introduce video surveillance information for abnormal tracing;

[0100] If it is necessary to conduct a video traceability necessity analysis, the second production power data corresponding to the abnormal monitoring result is used as the third production power data, and the equipment target corresponding to the third production power data is determined, and the video data of the equipment target is obtained, wherein the equipment target is a generating device and a data uploading device for the third production power data; the video data is a shot video of the equipment target;

[0101] Temporally aligning the third production power data and the video data of the equipment target, and generating a traceability progress, wherein temporally aligning the third production power data and the video data means developing the third production power data and the video data on the same time axis;

[0102] Determine the progress mark on the tracing progress, perform progress tracing back based on the preset tracing back speed, and obtain the facial features of the observer during the tracing back process, wherein the progress mark is the progress time point on the tracing progress, and the tracing data of the progress time point corresponding to the progress mark is displayed in real time in the dynamic display area on the visual dashboard; the preset tracing back speed is: the speed at which the progress mark is dragged back; the facial features are: the expression and facial features of the observer;

[0103] According to the feature matching results of the facial features of the observer and the preset progress mark determination features during the backtracking process, the backtracking results to be confirmed and the confirmation instructions of the backtracking results to be confirmed are generated; wherein the preset progress mark determination features are manually preset, such as: pupil dilation; the backtracking results to be confirmed are: when the facial features and the progress mark determination features match, the third production power data and video data at the corresponding progress mark corresponding to the marking time;

[0104] Obtain instruction reply information of the confirmation instruction according to the confirmation instruction of the backtracking result to be confirmed, wherein the confirmation instruction is an instruction for the observer to determine whether to save the backtracking result, such as a pop-up window "Possible backtracking results have been retrieved, do you confirm to store?";

[0105] According to the instruction reply information, the target backtracking result is determined and displayed on the visual dashboard. The instruction reply information includes confirmation and cancellation. When confirmed, the backtracking result to be confirmed is used as the target backtracking result; when canceled, the progress backtracking continues.

[0106] It should be noted that the present invention visualizes both the abnormal monitoring results and the tracing results of the abnormal monitoring results; specifically, the abnormal type is determined, and the necessity analysis of video tracing is performed. During the analysis, the corresponding relationship between the abnormal type and whether it is necessary to perform video tracing is manually preset; when it is necessary to perform abnormal analysis, the abnormal third production power data and its corresponding equipment target are determined, and video data is obtained; the third production power data and the video data are unfolded on the same time axis and a tracing progress is generated; there are progress marks on the tracing progress, and the progress mark at each position represents the progress information at a moment (the third production power data and video data at the time point corresponding to the progress mark). The observer checks the progress information. When an abnormality is found, the progress mark determination feature (such as pupil dilation) of the observer's face is extracted. Based on the progress mark at this time, a confirmation instruction of the tracing result to be confirmed is output. The tracing result to be confirmed is a system cache result. When the confirmation instruction is confirmed, the tracing result to be confirmed is stored in a read-only memory as a target tracing result, and is displayed based on a visual dashboard. The progress tracing process is more intuitive and intelligent.

[0107] From the above, it can be seen that the present invention provides a method and system for monitoring and visualizing the quality of production power data, which pre-processes the first production power data uploaded by each target enterprise to obtain the second production power data; introduces a data quality dynamic monitoring and analysis model to perform quality monitoring on the second production power data to obtain the data quality monitoring results and visualize them, thereby realizing the automatic uploading and automatic quality monitoring of production power data and improving the accuracy of subsequent data analysis.

[0108] Example 2

[0109] Reference Figures 2 to 4 As an embodiment of the present invention, a production power data quality monitoring and visualization system is provided, which specifically includes:

[0110] The data collection module 200 is used to collect the first power production data uploaded by each target enterprise, wherein the target enterprise is a coal industry enterprise; the first power production data is the data in the power production process uploaded by the target enterprise, including: enterprise power consumption, basic information, safety monitoring and personnel positioning;

[0111] The preprocessing module 300 is used to preprocess the first production power data and use the preprocessed data as the second production power data, wherein the second production power data is the preprocessed data, which has been free of errors and anomalies and has been standardized or normalized, and is more suitable for subsequent analysis;

[0112] A first acquisition module 400 is used to acquire a data quality dynamic monitoring and analysis model according to a target analysis process;

[0113] The second acquisition module 500 is used to perform quality monitoring on the second production power data based on the data quality dynamic monitoring and analysis model, and obtain the data quality monitoring results, wherein the quality monitoring of the second production power data based on the data quality dynamic monitoring and analysis model is: according to the data quality dynamic monitoring and analysis model, the stability and integrity of the access data are monitored in real time, and the monitoring and analysis results are displayed through a visualization page, and the data quality monitoring results (current transmission status, access data volume, number of access coal mining enterprises, data qualification rate and enterprise data stability analysis) are dynamically viewed;

[0114] The visualization display module 600 is used to visualize the data quality monitoring results.

[0115] In a feasible embodiment, the data collection module 200 includes:

[0116] The first production power data acquisition module 201 is used to acquire the first production power data through a data sharing platform; wherein the data sharing platform is based on a big data distributed network storage architecture.

[0117] It should be noted that the data sharing platform in this embodiment adopts a big data distributed network storage architecture, which stores a large number of data elements in a dispersed manner on multiple nodes according to the type of data, transmission frequency, etc., thereby improving the scalability of the metadata management architecture, estimating the number of nodes required based on the amount of data and the workload of calculation, and dynamically migrating data between nodes to achieve load balancing; at the same time, when a node fails, the data can be restored through a copy mechanism, which has less impact on upper-level applications.

[0118] In a feasible embodiment, the pre-processing module 300 includes:

[0119] Preprocessing the first production power data based on multiple preprocessing steps, wherein the preprocessing steps include data cleaning, data integration, data reduction and data conversion;

[0120] Specifically, data cleaning is to detect inconsistencies in the first production power data, identify noise data, filter and correct data, which can improve the consistency, accuracy, authenticity and availability of data quality; data integration is to integrate data from multiple data sources to form a centralized and unified database, which is conducive to improving the integrity, consistency, security and availability of big data quality; data induction is to reduce the size of the data set without compromising the accuracy of the analysis results, including dimension reduction, data reduction, data sampling, and improving the value density of big data, that is, improving the value of big data storage; data conversion is to process the first production power data based on rules or metadata, and based on models and learning, to achieve data unification and improve the consistency and availability of big data.

[0121] Furthermore, the second production power data is pre-processed data, which has been free of errors and anomalies and has been standardized or normalized, making it more suitable for subsequent analysis;

[0122] It should be noted that with the continuous advancement of the production process of coal mining enterprises, the data timeliness is high, the data differences between enterprises are large, the data quality is uneven, and the platform data collection itself has many defects. In addition, the data is easily affected by noise data, missing data values ​​and data conflicts during the push, transmission and reception process. Therefore, the present invention processes the collected first production power data based on multiple preprocessing links to obtain the second production power data, which greatly improves the data quality and the accuracy of subsequent analysis.

[0123] In a possible embodiment, if Figure 3 As shown, the first acquisition module 400 includes:

[0124] The historical data review record acquisition module 401 is used to obtain the historical data review record; wherein the historical data review record is: the record of the manual review of the historical production power data in the past, and the record includes: the time of the review, the identity of the reviewer, the type of data reviewed and the quantity;

[0125] The target analysis process acquisition module 402 is used to determine the target analysis process of the first reviewer according to the historical data review record; wherein the first reviewer is: the identity of the reviewer determined by parsing the historical data review record; the target analysis process is: the first reviewer analyzes the transmission status, access data volume, number of accessed coal mining enterprises, data qualification rate and enterprise data stability according to the historical production power data and the production power data at the time of review;

[0126] The model training module 403 is used to train the data quality dynamic monitoring analysis model according to the target analysis process. When training the data quality dynamic monitoring analysis model according to the target analysis process, the neural network model is used to learn the target analysis process to obtain an AI model that assists manual dynamic monitoring of data quality. The learning process of the neural network model is: the target analysis process is used as training data for model training until the model training reaches a convergence state.

[0127] It should be noted that the present embodiment introduces historical data review records, determines the target analysis process of the first reviewer, and trains a dynamic monitoring and analysis model for data quality according to the target analysis process, which is more reasonable.

[0128] Specifically, the target analysis process acquisition module 402 includes:

[0129] The access purpose determination submodule 4021 is used to determine the access purpose of the first access party according to the historical data access record; wherein the access purpose is the purpose or use of the historical data being accessed, including: data reference, data quality comparison and data verification;

[0130] The retrieval process determination submodule 4022 is used to determine the retrieval process for which the retrieval purpose is data quality comparison; wherein the retrieval process is the corresponding record retrieval process when the retrieval purpose is data quality comparison;

[0131] The target analysis process acquisition submodule 4023 is used to analyze the access process and acquire the target analysis process, wherein the target analysis process is a data quality comparison process obtained by analyzing the historical data access records.

[0132] Among them, the target analysis process acquisition submodule 4023 includes:

[0133] A dispatching authority acquisition unit is used to acquire the dispatching authority of the first access party; wherein the dispatching authority is: the authority of the first access party to view and access various data on the production power data detection platform;

[0134] A second reviewer determination unit, configured to determine the corresponding first reviewer as the second reviewer if the scheduling authority includes the review of historical data review records;

[0135] The preceding operation information acquisition unit is used to acquire the preceding operation information before the access time when the second access party accesses the corresponding historical data access record; wherein the preceding operation information is: the operation information of the second access party on the production power data detection platform before the access time, such as: which interface to view first and which interface to view next;

[0136] A first determination unit for access purpose, used to predict the access process according to the previous operation information, obtain the prediction result of the access process, and use it as the access purpose of the second access party;

[0137] A third reviewer determination unit, configured to determine the corresponding first reviewer as the third reviewer if the scheduling authority does not include the review of historical data review records;

[0138] A record form acquisition unit is used to acquire a request form for a third party to access the corresponding historical data access record; wherein the request form is a form that needs to be filled in by a person who does not have access rights to the historical data access record when conducting the corresponding access;

[0139] The second access purpose determination unit is used to parse the request form and obtain the access purpose of the third access party.

[0140] Specifically, the model training module 403 includes:

[0141] The first analysis speech acquisition submodule 4031 is used to analyze the target analysis process and acquire the first analysis speech; wherein the first analysis speech is the conversation information of the personnel in the analysis process;

[0142] The analysis screen extraction condition determination module 4032 is used to determine whether the speech time of the first analysis speech meets the analysis screen extraction condition according to the first analysis speech; wherein the analysis screen extraction condition is a determination condition for whether to extract the kanban interface being analyzed by the corresponding personnel, for example: if the first analysis speech contains the preset viewing semantics of the visualization kanban, the extraction of the analysis screen is triggered;

[0143] The second analysis speech extraction submodule 4033 is used to extract the analysis picture and the second analysis speech if the conditions are met; wherein the second analysis speech is: the first analysis speech after the speech time, and the speech time interval between the speech items of the second analysis speech is less than a preset speech time interval threshold (for example: 4 seconds);

[0144] The analysis content stator module 4034 is used to determine the analysis content items according to the sight line characteristics of the analysis picture; wherein the sight line characteristics are: the sight line landing point and the length of stay of the analysis picture; the analysis content items are: the data items analyzed on the analysis picture;

[0145] A model training submodule 4035 is used to train a data quality dynamic monitoring analysis model based on the second analysis speech and the analysis content item and according to a preset AI analysis model;

[0146] Among them, the model training submodule 4035 includes:

[0147] A time information alignment unit, used to expand both the analysis content item and the analysis speech item of the second analysis speech on the time axis; wherein, when expanding, the analysis content item and the analysis speech item correspond to the time points of the corresponding analysis / corresponding speech on the time axis one by one;

[0148] a target analysis content item determination unit, configured to traverse each analysis content item in sequence and take the analysis content item being traversed as the target analysis content item;

[0149] A time point determination unit, used to determine a first time point of a target analysis content item on the time axis and a second time point of an analysis content item next to the target analysis content item on the time axis;

[0150] A speech item acquisition unit to be matched, used to acquire the analysis speech item between the first time point and the second time point on the time axis and use it as the speech item to be matched;

[0151] The associated semantic item acquisition unit is used to acquire the associated semantic item of the target analysis content item; wherein the associated semantic item is: the semantics related to the analysis of the target analysis content item, which is preset manually, for example: the target analysis content item is: the stability of the production power data uploaded by enterprise A, then the associated semantic items are: the description semantics of enterprise A, the upload period semantics of the production power data uploaded, etc.;

[0152] A target analysis speech item matching unit is used to match the speech item to be matched with the associated semantic items in sequence, and if there is a match, determine the target analysis speech item, and associate the target analysis speech item with the target analysis content item;

[0153] The model training unit is used to train the data quality dynamic monitoring analysis model based on the target analysis content items and the corresponding associated target analysis speech items and according to the preset AI analysis model after all analysis content items have been traversed.

[0154] In a possible implementation, Figure 4 As shown, the visual display module 600 includes:

[0155] The abnormal monitoring result determination module 601 is used to determine the abnormal monitoring result in the data quality monitoring result; wherein the abnormal monitoring result is the production power data with abnormal data quality, such as: a certain power data missing at a certain time of enterprise A;

[0156] The visualization display module 602 is used to visualize the abnormal monitoring results and the tracing results of the abnormal monitoring results, wherein the tracing results are the tracing data that lead to the abnormal monitoring results.

[0157] Specifically, the visual display module 602 includes:

[0158] The first display submodule 6021 is used to display the abnormal monitoring results on a visual dashboard; wherein the visual dashboard is a graphical interface for displaying production power data and monitoring results;

[0159] The video tracing necessity analysis submodule 6022 is used to perform video tracing necessity analysis according to the abnormality type of the abnormal monitoring result; wherein the abnormality type is: abnormality type, such as: unstable data upload, data missing, etc.; the video tracing necessity analysis is: whether it is necessary to introduce video monitoring information for abnormal tracing;

[0160] The third production power data determination submodule 6023 is used to use the second production power data corresponding to the abnormal monitoring result as the third production power data if necessary;

[0161] The video data acquisition submodule 6024 is used to determine the device target corresponding to the third production power data and acquire the video data of the device target; wherein the device target is: a device for generating the third production power data and a device for uploading data; the video data is: a video shot of the device target;

[0162] The tracing progress generating submodule 6025 is used to align the third production power data and the video data in time and generate a tracing progress; wherein aligning the third production power data and the video data in time means developing the third production power data and the video data on the same time axis;

[0163] The facial feature acquisition submodule 6026 is used to determine the progress mark on the tracing progress, perform progress tracing based on the preset tracing speed, and obtain the facial features of the observer during the tracing process; wherein the progress mark is the progress time point on the tracing progress, and the tracing data of the progress time point corresponding to the progress mark is displayed in real time in the dynamic display area on the visual dashboard; the preset tracing speed is: the speed at which the progress mark is dragged back; the facial features are: the expression and facial features of the observer;

[0164] The to-be-confirmed retrospective result confirmation submodule 6027 is used to generate the to-be-confirmed retrospective result according to the feature matching result of the facial feature and the preset progress mark determination feature; wherein the preset progress mark determination feature is manually preset, such as: pupil dilation; the to-be-confirmed retrospective result is: when the facial feature and the progress mark determination feature match, the third production power data and video data at the corresponding progress mark corresponding to the marking time;

[0165] The instruction reply information acquisition submodule 6028 is used to generate a confirmation instruction for the backtracking result to be confirmed, and obtain instruction reply information for the confirmation instruction; wherein the confirmation instruction is an instruction for the observer to determine whether to save the backtracking result, such as a pop-up window "Possible backtracking results have been retrieved, do you confirm to store?";

[0166] The second display submodule 6029 is used to determine the target backtracking result according to the instruction reply information, and display the target backtracking result on the visual dashboard. The instruction reply information includes confirmation and cancellation. When confirmed, the backtracking result to be confirmed is used as the target backtracking result; when cancelled, the progress backtracking continues.

[0167] It should be noted that the present invention visualizes both the abnormal monitoring results and the tracing results of the abnormal monitoring results; specifically, the abnormal type is determined, and the necessity analysis of video tracing is performed. During the analysis, the corresponding relationship between the abnormal type and whether it is necessary to perform video tracing is manually preset; when it is necessary to perform abnormal analysis, the abnormal third production power data and its corresponding equipment target are determined, and video data is obtained; the third production power data and the video data are unfolded on the same time axis and a tracing progress is generated; there are progress marks on the tracing progress, and the progress mark at each position represents the progress information at a moment (the third production power data and video data at the time point corresponding to the progress mark). The observer checks the progress information. When an abnormality is found, the progress mark determination feature (such as pupil dilation) of the observer's face is extracted. Based on the progress mark at this time, a confirmation instruction of the tracing result to be confirmed is output. The tracing result to be confirmed is a system cache result. When the confirmation instruction is confirmed, the tracing result to be confirmed is stored in the read-only memory as the target tracing result, and is displayed based on the visual dashboard. The progress tracing process is more intuitive and intelligent.

[0168] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0169] Example 3

[0170] The present embodiment provides a computing device, which includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for monitoring and visualizing the quality of production power data is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0171] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method proposed in the above embodiment is implemented.

[0172] The storage medium proposed in this embodiment and the method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0173] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccess Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0174] Example 4

[0175] Reference Figure 5 This embodiment provides an application example of a method and system for monitoring and visualizing production power data quality, in order to verify and illustrate the technical effects adopted in this method.

[0176] Collect the total electricity consumption of the target coal mining enterprise; the total electricity consumption is the electricity consumption of the target coal mining enterprise, for example: 138,390 kWh;

[0177] Collect equipment data of production equipment of target coal mining enterprises; the equipment data includes: wind tunnel sensor data and wind tunnel wind speed sensor data;

[0178] Quality monitoring is carried out based on the total power consumption and equipment data, and the quality monitoring results are visualized. Among them, for the total power consumption, the quality monitoring is the number of degrees of electricity consumed; for the wind tunnel wind duct sensor data, the quality monitoring is the wind tunnel status; for the wind tunnel wind speed sensor data, the quality monitoring is the wind speed.

[0179] From the above, it can be seen that the present invention provides a method and system for monitoring and visualizing the quality of production power data, which pre-processes the first production power data uploaded by each target enterprise to obtain the second production power data; introduces a data quality dynamic monitoring and analysis model to perform quality monitoring on the second production power data to obtain the data quality monitoring results and visualize them, thereby realizing the automatic uploading and automatic quality monitoring of production power data and improving the accuracy of subsequent data analysis.

[0180] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0181] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0182] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0183] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0185] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0186] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for monitoring and visualizing production power data quality, characterized in that: include: Collect the first production power data uploaded by each target enterprise; preprocessing the first production power data, and using the preprocessed data as second production power data; Acquire a dynamic monitoring and analysis model for data quality based on the target analysis process; Based on the data quality dynamic monitoring and analysis model, quality monitoring is performed on the second production power data, and a data quality monitoring result is obtained; The data quality monitoring results are displayed visually.

2. The method for monitoring and visualizing production power data quality according to claim 1, characterized in that: The collecting of the first production power data uploaded by each target enterprise includes: The first production power data uploaded by each target enterprise is collected through a data sharing platform, wherein the data sharing platform is based on a big data distributed network storage architecture.

3. The method for monitoring and visualizing production power data quality according to claim 2, characterized in that: Preprocessing the first production power data includes: Preprocessing the first production power data based on multiple preprocessing steps, wherein the preprocessing steps include data cleaning, data integration, data reduction and data conversion; The data cleaning is to detect inconsistencies, identify noise data, filter and correct the first production power data; the data integration is to integrate data from multiple data sources to form a centralized and unified database; the data summarization is to reduce the size of the data set without compromising the accuracy of the analysis results, including dimensionality reduction, data reduction, and data sampling; the data conversion is to process the first production power data based on rules or metadata, and based on models and learning to achieve data unification.

4. The method for monitoring and visualizing production power data quality according to claim 3, characterized in that: The acquisition of a data quality dynamic monitoring and analysis model according to the target analysis process includes: Obtain historical data review records; Determining the target analysis process of the first reviewer according to the historical data review record; The target analysis process is to determine the purpose of the first reviewer's review based on the historical data review records, and to determine and analyze the review process in which the purpose of the review is data quality comparison, that is, the first reviewer analyzes the transmission status, access data volume, number of accessed coal mining enterprises, data qualification rate and enterprise data stability based on the historical production power data and the production power data at the time of review; According to the target analysis process, a data quality dynamic monitoring and analysis model is trained.

5. The method for monitoring and visualizing production power data quality according to claim 4, characterized in that: According to the target analysis process, the training data quality dynamic monitoring and analysis model includes: The target analysis process obtains a first analysis statement; According to the first analysis speech, determining whether the speech time of the first analysis speech meets the analysis screen extraction condition; If the analysis screen extraction condition is met, the analysis screen and the second analysis speech are extracted; Determining analysis content items according to the sight line features of the analysis picture; Based on the second analysis speech, the analysis content item and the preset analysis model, a data quality dynamic monitoring analysis model is trained.

6. The method for monitoring and visualizing production power data quality according to claim 4, characterized in that: The target analysis process also includes: Obtaining the scheduling authority of the first reviewer; If the scheduling authority of the first reviewer includes the review of historical data review records, the corresponding first reviewer will be used as the second reviewer; Acquire the preceding operation information before the time at which the second reviewer reviews the corresponding historical data review record; Predicting the review process according to the preceding operation information, obtaining the review process prediction result, and using it as the review purpose of the second reviewing party; If the scheduling authority of the first reviewer does not include the review of historical data review records, the corresponding first reviewer will be regarded as the third reviewer; A request form for the third reviewer to review the corresponding historical data review record is obtained, and the request form is parsed to obtain the review purpose of the third reviewer.

7. The method for monitoring and visualizing production power data quality according to claim 6, characterized in that: Visualizing the data quality monitoring results includes: Display abnormal monitoring results on a visual dashboard; Performing a necessity analysis of video tracing according to the abnormal type of the abnormal monitoring result; If it is necessary to perform a video tracing necessity analysis, the second production power data corresponding to the abnormal monitoring result is used as the third production power data, and the equipment target corresponding to the third production power data is determined, and the video data of the equipment target is obtained; Temporally aligning the third production power data with the video data of the device target, and generating a retrospective progress; Determine a progress mark on the tracing progress, perform progress tracing back based on a preset tracing back speed, and obtain facial features of an observer during the tracing back process; Determine the feature matching result of the features according to the facial features of the observer and the preset progress mark in the backtracking process, generate the backtracking result to be confirmed and the confirmation instruction of the backtracking result to be confirmed; Obtaining instruction reply information of the confirmation instruction according to the confirmation instruction of the backtracking result to be confirmed; According to the instruction reply information, a target backtracking result is determined, and the target backtracking result is displayed on a visual dashboard.

8. A system using the method for monitoring and visualizing production power data quality as claimed in any one of claims 1 to 7, characterized in that: include: A data collection module (200), used for collecting first production power data uploaded by each target enterprise; A preprocessing module (300), used for preprocessing the first production power data, and using the preprocessed data as second production power data; A first acquisition module (400) is used to acquire a data quality dynamic monitoring analysis model according to a target analysis process; A second acquisition module (500) is used to perform quality monitoring on the second power production data based on the data quality dynamic monitoring and analysis model, and obtain data quality monitoring results; The visual display module (600) is used to visually display the data quality monitoring results.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images