Yield data analysis method and related equipment

Abstract

The invention discloses a yield data analysis method and related equipment, and relates to the technical field of the steel industry, the method comprises the steps that yield data of corresponding procedures of all operation parts are obtained, and the operation parts comprise a hot rolling operation part, an automobile plate operation part and a silicon steel operation part; based on the yield data, utilizing a big data platform to construct an off-line data warehouse, carrying out data preprocessing, and generating a source pasting data layer; on the basis of the source data layer, reconstructing a logic calculation process to form a basic theme layer; based on the basic theme layer, constructing a multi-dimensional data mart, and performing multi-dimensional analysis on the yield data to obtain an analysis result; and based on the analysis result, constructing a yield index map so as to visually display the index relationship.

Description

Technical Field

[0001] The present application relates to the technical field of the steel industry, and in particular to a yield rate data analysis method and related equipment. Background Art

[0002] In modern manufacturing processes, yield rate is an important indicator for measuring production efficiency and product quality. Yield rate data analysis mainly relies on calculating and comparing the yield rates of each process in order to promptly identify problems in the production process and optimize the production process. However, traditional yield rate data analysis methods often fail to deeply explore the relationship between each process and lack effective integration and analysis of multi-dimensional data. This results in a relatively simple data presentation method in the process of yield rate optimization, which makes it difficult to fully reflect the impact of various production factors on the yield rate.

[0003] To solve these problems, traditional methods usually evaluate based on a single yield rate data, ignoring the mutual influence of various process parameters, equipment status, environmental conditions and other factors in the production process. This makes the analysis results too rough and cannot provide sufficient information for production line optimization and decision-making. Therefore, there is an urgent need for a yield rate data analysis method that can analyze the yield rate data more comprehensively and accurately and provide more efficient decision support. Summary of the invention

[0004] A series of simplified concepts are introduced in the Summary of the Invention section, which will be further described in detail in the Detailed Description of the Invention section. The Summary of the Invention section of this application does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.

[0005] In a first aspect, the present application provides a method for analyzing yield rate data, comprising:

[0006] Obtaining the yield rate data of the corresponding process of each operation section, wherein the above-mentioned operation section includes the hot rolling operation section, the automobile sheet operation section and the silicon steel operation section;

[0007] Based on the above-mentioned yield rate data, an offline data warehouse is built using the big data platform to perform data preprocessing and generate a source data layer;

[0008] Based on the above-mentioned source data layer, reconstruct the logical calculation process to form the basic theme layer;

[0009] Based on the above basic theme layer, a multidimensional data mart is constructed to conduct multidimensional analysis on the above yield rate data to obtain the analysis results;

[0010] Based on the above analysis results, a yield rate index map is constructed to visualize the indicator relationship.

[0011] In a feasible implementation manner, the above data preprocessing step includes:

[0012] The above-mentioned yield rate data is extracted in an incremental manner based on the offline data warehouse of the above-mentioned big data platform, and outliers are eliminated and cleaned for the above-mentioned yield rate data.

[0013] In a feasible implementation, the offline data warehouse is a data warehouse supporting column storage built on Hadoop and Hive platforms.

[0014] In a feasible implementation, it also includes: optimizing the above-mentioned source data layer using a Kudu database and / or ClickHouse database based on column storage.

[0015] In a feasible implementation manner, the above reconstruction logic calculation process specifically includes:

[0016] Based on the Impala computing engine, a distributed computing method is used to perform logical calculation and index analysis on the above yield rate data to obtain optimized yield rate data, wherein the above logical calculation is achieved through hierarchical processing.

[0017] In a feasible implementation manner, the multidimensional data mart is constructed, and the yield rate data is analyzed in multiple dimensions to obtain analysis results, including:

[0018] Based on Kylin technology, pre-calculate the indicators after different dimensional combinations and store them in the data cube, where the above different dimensional combinations are determined based on the above yield rate data and new dimensional data;

[0019] Based on the above data cube, a multi-dimensional analysis is performed to obtain the analysis results.

[0020] In a feasible implementation, based on the above analysis results, a yield rate index map is constructed to visualize the index relationship, including:

[0021] Based on the above analysis results, establish the relationship link between indicators and construct the yield rate indicator map;

[0022] According to the indicator data of the yield rate indicator map, marking the completion status of the indicator, wherein the indicator data includes the planned value and the actual completion value;

[0023] When the actual completion value is less than the planned value, the influencing factors are displayed.

[0024] In a second aspect, the present application proposes a yield rate data analysis device, comprising:

[0025] A yield rate data acquisition unit is used to acquire the yield rate data of the corresponding process of each operation section, wherein the above-mentioned operation section includes a hot rolling operation section, an automobile sheet operation section and a silicon steel operation section;

[0026] A source data layer generation unit is used to build an offline data warehouse based on the above-mentioned yield rate data using a big data platform, perform data preprocessing, and generate a source data layer;

[0027] A basic theme layer forming unit, used to reconstruct the logic calculation process based on the above-mentioned post source data layer to form a basic theme layer;

[0028] A multi-dimensional data analysis unit is used to construct a multi-dimensional data mart based on the basic subject layer, perform multi-dimensional analysis on the yield rate data, and obtain analysis results;

[0029] The indicator relationship analysis unit is used to construct a yield rate indicator map based on the above analysis results to visualize the indicator relationship.

[0030] In a third aspect, an electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the yield rate data analysis method of any one of the first aspects when executing the computer program stored in the memory.

[0031] In a fourth aspect, the present application further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the yield rate data analysis method of any one of the first aspects.

[0032] In summary, the embodiment of the present application adopts multi-dimensional data analysis, which can comprehensively analyze the yield rate data of each process from multiple angles, not just relying on traditional single-dimensional analysis. This multi-dimensional analysis method not only improves the accuracy of data analysis, but also makes the evaluation of yield rate data more comprehensive, and better reveals the potential problems in the production process. Through Kylin technology and data cube, the present application can pre-calculate and store the yield rate index after combining each analysis dimension, which improves the efficiency of query and analysis. The data cube supports fast multi-dimensional query, thereby realizing real-time analysis and dynamic query of yield rate data, avoiding the inefficiency caused by repeated calculations in traditional methods. In addition, the yield rate index map constructed in combination with visualization technology enables production managers to clearly see the relationship and influence path between each index through intuitive graphical display. Through the index map, the key factors and influencing links in the production process can be clearly presented, helping managers to quickly identify bottlenecks and key influencing factors in the production process, so as to make timely adjustments and decisions. When performing data analysis, this application can mark the completion status of indicators based on the comparison between planned values ​​and actual values, and automatically display relevant influencing factors when it is found that the actual value deviates from the planned value, providing effective support for abnormal monitoring in the production process, helping enterprises to better understand the key factors in production, optimize process flow, and improve production efficiency and product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present specification. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0034] Figure 1 A schematic diagram of a method for analyzing yield rate data provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the structure of a yield rate data analysis device provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of an electronic device for analyzing yield rate data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0038] See also Figure 1 , is a schematic flow chart of a method for analyzing yield rate data provided in an embodiment of the present application, which may specifically include:

[0039] S110, obtaining the yield rate data of the corresponding process of each operation section, wherein the above-mentioned operation sections include the hot rolling operation section, the automobile sheet operation section and the silicon steel operation section;

[0040] Exemplarily, first obtain the yield rate data of the corresponding process from each operation department. Specifically, the operation department includes the hot rolling operation department, the automobile plate operation department and the silicon steel operation department, etc. These processes represent different stages in the production process. The yield rate is a key indicator for measuring production efficiency and quality, and the ratio between the actual number of qualified products and the total production quantity is calculated. By collecting the production data of each process (such as hot rolling, automobile plates and silicon steel, etc.), the yield rate data corresponding to each process can be obtained, which provides a basis for subsequent analysis.

[0041] Data sources include production monitoring systems, quality inspection systems, and real-time production data records for each process. Through these data sources, the system can automatically extract data related to yield rate, thereby providing basic data for subsequent data analysis, anomaly detection, and optimization. The yield rate data of each process reflects the production efficiency and quality performance of that link, and is a key performance indicator in the entire production chain.

[0042] S120, based on the above-mentioned yield rate data, using the big data platform to build an offline data warehouse, perform data preprocessing, and generate a source data layer;

[0043] For example, using big data technologies (such as Hadoop, Spark, etc.) to build a data warehouse can effectively process and store massive amounts of production data. As a data management system, the data warehouse can uniformly store data from different sources and provide efficient data processing capabilities. Through the big data platform, it is easy to process yield rate data from different processes and links, and ensure efficient storage and fast access to data.

[0044] At this stage, the yield rate data and other related data are pre-processed and stored in the offline data warehouse. The offline data warehouse is used to store historical data and regularly collected data, and it can provide a stable storage environment for subsequent batch data analysis. Offline processing can reduce the burden of real-time data processing and make the processing and analysis of large-scale data more efficient.

[0045] The generated source data layer is the lowest level in the data warehouse for storing source data, which usually includes original data and data after simple processing. In this layer, the yield rate data is stored according to the structure of the source system, retaining the original attributes of the data. The source data layer provides a basis for data analysis and processing in the upper layer, ensuring that the data in the subsequent processing process is not lost or deformed, and can be traced back to the source of the original data.

[0046] S130, based on the above-mentioned post source data layer, reconstruct the logical calculation process to form a basic theme layer;

[0047] Exemplarily, the source data layer contains the original yield rate data that has been initially cleaned and converted, which provides a reliable data source for subsequent data processing. By processing based on the source data layer, the consistency, completeness and accuracy of the data are ensured, laying the foundation for further calculation and analysis.

[0048] The reconstruction logic calculation process refers to further mathematical calculations and logical deductions of the original data. This process usually includes data aggregation, application of calculation formulas, and processing of relationships between multiple data dimensions. Through logical calculations, the system can extract more valuable information and indicators from the source data, such as extracting the yield calculation results under different processes, time periods, process parameters and other dimensions from the original yield data. For example, the calculation of the yield involves weighted averaging of data from different processes, or calculating the trend changes of the yield according to different time periods, which are all part of the reconstruction calculation process.

[0049] The basic subject layer is a structured layer formed by dividing the data into themes based on the reconstruction of the calculation process. The formation of the subject layer is to classify and organize the data according to business areas, functional modules or calculation targets. For example, the yield rate data can be classified according to themes such as production processes (such as hot rolling, cold rolling) and process parameters (such as temperature, pressure). By organizing the data in layers and themes, the basic subject layer can provide a clear logical structure for the subsequent construction of multidimensional data marts, which facilitates flexible query and analysis of data of different themes during the analysis process.

[0050] S140, based on the basic subject layer, construct a multidimensional data mart, perform multidimensional analysis on the yield rate data, and obtain analysis results;

[0051] For example, in the above steps, the data has been thematically classified and structured by reconstructing the logical calculation process and generating the basic theme layer. The data in the basic theme layer has been clearly divided according to business areas and analysis objectives, providing clear structural support for the subsequent multidimensional data mart construction. The basic theme layer ensures that the classification and processing of data are consistent and accurate, providing a reliable data source for multidimensional analysis.

[0052] A multidimensional data mart is a subset data warehouse for specific analysis needs. It usually slices, aggregates and stores data according to different dimensions (such as time, production process, equipment, process parameters, etc.). By building a multidimensional data mart, the system can perform efficient multidimensional analysis of yield rate data.

[0053] The construction of a multidimensional data mart usually relies on a data cube. In this structure, the yield rate data will be divided into multiple dimensions, and users can perform cross-queries and aggregations between different dimensions. Through a multidimensional data mart, the yield rate data can be analyzed from multiple dimensions. Common dimensions include the time dimension (analyzing the yield rate trend in different time periods), the process dimension (for example, analyzing the yield rate performance under different processes such as hot rolling and cold rolling), and the equipment dimension (analyzing the yield rate differences under different production equipment or production lines). This multidimensional analysis method can help to gain an in-depth understanding of the impact of each production link, process, and equipment on the yield rate, and provide data support for optimizing the production process.

[0054] By querying and analyzing the multidimensional data mart, the system can obtain analysis results, which usually include the changing trend of yield rate in different dimensions, the performance of yield rate of each process or each equipment, and the impact of different process parameters on yield rate. These analysis results can be used for decision support, for example, to identify key factors affecting yield rate, predict production bottlenecks, or provide directions for production optimization.

[0055] S150. Based on the above analysis results, a yield rate index map is constructed to visualize the index relationship.

[0056] For example, the analysis results are used as input to provide data support for the construction of a yield rate indicator map; the yield rate indicator map is a visualization tool that displays the relationship between the yield rate and its related indicators in the form of a chart or network diagram. This map can intuitively display the interactions and dependencies between various indicators (such as the yield rate of different processes, process parameters, etc.). Through this map, managers can clearly see which indicators have a significant impact on the yield rate and which factors may be the root cause of inefficiency or substandard quality in the production process.

[0057] In the yield rate indicator map, the cause-effect relationship and mutual influence links between indicators will be displayed. For example, if equipment failure in a certain production link leads to a decrease in production efficiency, which in turn affects the yield rate, this process will be clearly presented in the indicator map through connecting lines or arrows. This visual method helps production managers quickly identify the reasons for fluctuations in the yield rate and make timely adjustments and decisions.

[0058] In some examples, the data preprocessing step includes:

[0059] The above-mentioned yield rate data is extracted in an incremental manner based on the offline data warehouse of the above-mentioned big data platform, and outliers are eliminated and cleaned for the above-mentioned yield rate data.

[0060] For example, the incremental method is used to extract the yield rate data from the offline data warehouse of the big data platform. The application of the incremental method can ensure that the system only processes the newly generated data each time, rather than processing all the data from scratch each time, which can greatly improve the efficiency of data processing and avoid repeated calculations and unnecessary resource consumption.

[0061] The extracted yield rate data needs to be processed by outlier removal and data cleaning. Outlier removal is a key step in data preprocessing. The purpose is to identify and remove extreme values ​​that deviate significantly from the majority of the data. These outliers may come from errors in the data collection process, equipment failures, or human operation problems. If these outliers are not removed, they may affect the accuracy of subsequent analysis results and lead to erroneous conclusions.

[0062] Data cleaning includes processing missing data, duplicate data, and erroneous data to ensure data integrity, consistency, and accuracy. The cleaned data will become a reliable input for subsequent data analysis and decision support.

[0063] In some instances, Hadoop and Hive platforms are used to build offline data warehouses that support columnar storage.

[0064] For example, Hadoop is a powerful distributed computing framework that can process massive amounts of data. Through its efficient data processing capabilities, it provides support for the construction of data warehouses. Hive is a data warehouse system based on Hadoop that can provide a SQL-style query interface, making data storage and query more convenient. In this offline data warehouse, column storage is used in the data storage structure, mainly to improve the efficiency of large-scale data query and analysis. Compared with traditional row storage, column storage is more efficient when reading data, especially for analytical queries, because column storage can only load the required column data when querying, without reading the entire row of data. This is particularly important for storing multi-dimensional, large-scale data sets such as yield rate data, which can greatly reduce the time for data scanning and improve query performance.

[0065] The offline data warehouse, through the combination of Hadoop and Hive platforms, can efficiently store and manage massive yield rate data, and provide strong support for subsequent data extraction, preprocessing and analysis. The design of this architecture not only improves data processing efficiency, but also ensures data scalability and maintainability, meeting the high-performance requirements for data storage in big data analysis scenarios.

[0066] In some examples, it also includes:

[0067] The above-mentioned source data layer is optimized using Kudu database and / or ClickHouse database based on column storage.

[0068] For example, Kudu and ClickHouse are both high-performance columnar databases, specially designed for big data analysis and high-concurrency queries, which can greatly improve the efficiency of data storage and processing. Kudu is a columnar storage database for big data processing, which is particularly suitable for scenarios that require efficient writing and real-time analysis. Kudu supports complex joint primary key design, allowing the dimensions involved in the query to be used as primary keys, which makes data more efficient when querying. Its multi-partition feature can also provide a more efficient retrieval path to ensure a fast response to query operations. Therefore, using Kudu to optimize the source data layer can significantly improve the efficiency of data retrieval and analysis, especially in the scenario of processing yield data, it can quickly extract the required data for multi-dimensional analysis.

[0069] ClickHouse is a column-based database designed for real-time analytical queries, and is particularly good at handling high-concurrency queries on large data sets. ClickHouse is widely used in data warehouses and OLAP (online analytical processing) systems for its extremely high query speed and optimized column-based storage. In this application, ClickHouse is used to optimize the source data layer, which can achieve real-time data analysis, support rapid aggregation and analysis of yield rate data in the production process, provide instant feedback, and help managers make quick decisions.

[0070] Through the combined use of Kudu and ClickHouse, the data storage structure of the source data layer has been significantly optimized, and the query performance has been greatly improved. The advantages of column storage are reflected in efficient data compression and high-speed query. At the same time, both Kudu and ClickHouse support distributed computing, which can ensure the scalability and stability of the system in a massive data environment.

[0071] In some examples, the above-mentioned reconstruction logic calculation process includes:

[0072] Based on the Impala computing engine, a distributed computing method is used to perform logical calculation and index analysis on the above yield rate data to obtain optimized yield rate data, wherein the above logical calculation is achieved through hierarchical processing.

[0073] For example, Impala is a high-performance distributed computing engine designed for big data analysis, supporting data storage and query in Hadoop systems. By adopting the Impala computing engine, parallel computing of large-scale yield rate data can be achieved. The distributed computing method allows data processing tasks to be distributed to multiple computing nodes, avoiding the performance bottlenecks encountered in traditional single-machine computing, and significantly improving the computing speed and processing power. In the process of yield rate analysis, Impala makes parallel processing and real-time computing of data possible, especially when processing complex multi-dimensional data, it can quickly aggregate yield rate data under different processes, time periods and process parameters, ensuring the efficiency and real-time nature of data analysis.

[0074] In-depth analysis and integration of the indicator logic of yield rate analysis is carried out to clarify the calculation method of each indicator and its relationship with other indicators. For example, the yield rate is not just a ratio indicator, but may be related to multiple factors such as production process, equipment performance, and process parameters. Therefore, indicator logic analysis and integration helps optimize the calculation process by identifying these relationships, making the calculation of the yield rate more scientific and accurate.

[0075] Through hierarchical processing, the calculation process is divided into multiple levels, and each level processes different types of data or calculation tasks. This hierarchical strategy can not only optimize the calculation process and avoid unnecessary repeated calculations, but also improve the flexibility and scalability of the calculation. The calculation tasks of each layer can be executed in parallel, making the overall calculation more efficient. For example, you can first process basic data in one level, and then perform complex statistical analysis or model application in a higher level, and finally get the optimized yield rate result.

[0076] In some examples, the multidimensional data mart is constructed to perform multidimensional analysis on the yield rate data to obtain analysis results, including:

[0077] Based on Kylin technology, pre-calculate the indicators after different dimensional combinations and store them in the data cube, where the above different dimensional combinations are determined based on the above yield rate data and new dimensional data;

[0078] Based on the above data cube, a multi-dimensional analysis is performed to obtain the analysis results.

[0079] For example, when building a multidimensional data mart, Kylin technology is used to pre-calculate the yield rate indicators under different dimensional combinations. Kylin is an OLAP (online analytical processing) engine based on Hadoop, which can perform fast multi-dimensional analysis on massive data. Through Kylin's data cube technology, the system will combine different dimensional combinations from the yield rate data and new dimensional data (such as time, process, equipment, production batch, etc.), and pre-calculate the indicators under these combinations.

[0080] Pre-calculated indicators are stored in the data cube, so that when querying, the calculated results can be directly extracted from the cube without the need for real-time recalculation, which significantly improves query efficiency.

[0081] The pre-calculated results stored in the data cube can be used to mine the inherent rules of the yield rate data through multi-dimensional analysis. Based on the data cube, users can flexibly query and analyze from multiple dimensions.

[0082] In this way, the system can quickly generate the performance of the yield rate in different dimensions and reveal the impact of different dimensions on the yield rate. For example, users can query the performance of the yield rate in a specific time period, under certain process parameters, and under different equipment usage conditions to obtain more accurate analysis results. This analysis can not only help production managers monitor each production link in real time, but also provide data support for optimization decisions in the production process.

[0083] In some examples, based on the above analysis results, a yield rate index map is constructed to visualize the index relationship, including:

[0084] Based on the above analysis results, establish the relationship link between indicators and construct the yield rate indicator map;

[0085] According to the indicator data of the yield rate indicator map, marking the completion status of the indicator, wherein the indicator data includes the planned value and the actual completion value;

[0086] When the actual completion value is less than the planned value, the influencing factors are displayed.

[0087] For example, through multidimensional data analysis and data cube, detailed analysis results of the yield rate are constructed, revealing the impact of various production links, process parameters, equipment status, etc. on the yield rate. Based on these analysis results, the system further establishes the relationship link between indicators and clarifies the causal relationship or interdependence between the indicators.

[0088] Through the yield rate indicator map, these relationship links are visualized, allowing managers to intuitively see how the various indicators influence and transmit each other, helping to identify the key factors affecting the yield rate. For example, equipment failure → reduced production efficiency → reduced yield rate, such a causal link can be clearly presented in the indicator map.

[0089] Each indicator on the yield rate indicator map will be marked with its completion status, that is, by comparing the planned value and the actual completion value, it is shown whether the indicator is completed as expected. If the actual completion value is greater than or equal to the planned value, it means that the completion is good, the indicator has completed the corresponding plan, and is displayed normally, and you can choose to jump to the corresponding detailed report; otherwise, further analysis of the reasons is required. For example, the planned value of the yield rate may be a predetermined production target, while the actual completion value reflects the yield rate achieved in the actual production process. By comparing these two values, the system can determine whether the production target has been achieved.

[0090] When the actual completion value is less than the planned value, the system will automatically display possible influencing factors. These influencing factors are inferred based on previous analysis results and may be related to production processes, equipment operating conditions, raw material quality, etc. By marking these influencing factors, managers can quickly identify the specific reasons why the yield rate is lower than expected. For example, if the yield rate does not meet expectations, the system may display influencing factors such as equipment failure, improper temperature control, and human operating errors, etc., to help decision makers make targeted adjustments and optimizations.

[0091] The purpose of this invention is to meet the needs of business personnel for multi-dimensional analysis and rapid query of quality indicators with quality analysis users as the center. At the same time, through the information system, a clear KPI indicator map is built for managers to help determine plans and benchmarks that meet production reality, and to track and analyze the quality control level, so as to formulate an effective action plan.

[0092] Taking the spot-related indicators in the quality analysis of a steel plant as an example, the optimization process is divided into the following stages: current situation analysis, platform environment construction, source data layer optimization, basic theme layer optimization, application theme layer optimization and indicator map construction. The specific steps are as follows:

[0093] First, sort out each link of data logic processing, including its sequence, related database views, stored procedures and table structures, and clarify the blood relationship of data. Taking the yield rate analysis of a steel plant as an example, the yield rate management mainly covers three major processes: hot rolling operation department, automobile plate operation department and silicon steel operation department. In the past, the yield rate analysis of these processes was carried out in their respective PES systems, and only simple indicator statistics were performed, resulting in the data of each process not being able to communicate with each other, and centralized management and data analysis were not achieved.

[0094] Under the existing system, the steel plant built an offline data warehouse based on DB2 MPP (massive parallel processing), and extracted data from the business system through DATASTAGE and stored it in the base table of the source layer (ODS layer). Then, the data was summarized according to business needs to form the corresponding basic theme layer and application theme layer. However, the various storage processes in the system were complex and error-prone, and the execution time was long, which affected the overall data processing efficiency.

[0095] In addition, DB2 is a row-based storage database, which is mainly suitable for data writing and modification, and is suitable for simple data processing scenarios, but its performance is difficult to support the large number of detailed queries required for quality analysis. The multi-dimensional analysis architecture built on FineReport implements analysis through data sets (i.e. multi-dimensional aggregated indicators), but because data transmission depends on DB2, the execution time of the data set is long, which will slow down the response speed of the front-end indicator query and affect the real-time performance of the system.

[0096] In addition, the indicators constructed by the existing system are relatively scattered and have not formed a unified indicator architecture. At the same time, closed-loop management has not yet been formed in the system management process.

[0097] In combination with the characteristics of the big data platform, an offline data warehouse based on HIVE is used to extract database table data in an incremental manner through SQOOP and AZKB, and the extracted data is cleaned and processed to remove abnormal data, thereby reducing the pressure on the subsequent logical computing layer. By optimizing the big data platform architecture, building the Kylin and ClickHouse environment, the deployment of the big data platform infrastructure is realized.

[0098] Kylin is an OLAP engine based on Hadoop. It can pre-calculate and store the indicators under all dimension combinations that need to be analyzed in data cubes. The data cube provides efficient multi-dimensional analysis functions, which can quickly locate the dimension path during query, greatly improving the efficiency of analysis and query.

[0099] By building a big data platform environment of Kylin and ClickHouse, efficient infrastructure support is provided for the rapid analysis, processing and query of subsequent data.

[0100] In the process of optimizing the source data layer, data is extracted from the business system in an incremental manner, and only new or updated data is processed to reduce the amount of data processing. Data is deduplicated, missing values ​​are filled, and outliers are removed to ensure data accuracy and consistency. By cleaning and optimizing data, redundant data is reduced, the burden on the subsequent logical calculation layer is reduced, and calculation efficiency is improved. The optimized data is more streamlined, the query response speed is faster, and multi-dimensional analysis and fast report generation are supported.

[0101] In the optimization process of the basic theme layer, the storage process of logical calculation is reconstructed, the logic of various indicators is analyzed and integrated, and the calculation process is optimized through layered processing. Based on the Impala computing engine, batch tasks are executed in a distributed computing manner in the form of data calculation scripts, thereby improving data processing efficiency.

[0102] Based on Hive in the big data platform, a data warehouse was built, and the detailed application data layer was built using Kudu's columnar storage structure. The Kudu database is designed for analytical query scenarios and supports efficient queries. It has the ability to design complex joint primary keys, allowing multiple dimensions involved in the query to be indexed as primary keys, greatly improving query performance. In addition, Kudu's multi-partition feature further optimizes the data retrieval path, allowing faster responses when performing large-scale data queries.

[0103] By reconstructing the logical computing storage process, the data processing speed is further optimized. Script splitting and processing process optimization reduce the complexity and execution time of the storage process. Finally, the application of data warehouses built on the Hive database and Kudu / ClickHouse databases further improves the efficiency of data storage and query.

[0104] During the optimization of the application layer, a multidimensional data mart is established based on various dimensions to facilitate statistical aggregation of data. By adding data indicator dimensions based on the original data extraction, a detailed statistical analysis of the components and causes of the yield rate and process parameters is further carried out. For example, the impact of rolling line burnout on the yield rate is analyzed, including parameters such as furnace time, furnace entry temperature and furnace exit temperature; it also includes the trimming width analysis and head and tail cutting analysis of the cold rolling mill. Through these dimensions, the system can realize full-process data tracking from steelmaking, hot rolling to cold rolling, helping to deeply understand the changes in the yield rate and its influencing factors.

[0105] The purpose of building a yield rate indicator map is to help managers quickly obtain indicator information and clearly understand the relationship and links between indicators. By combining the big data platform, an indicator map integrating performance visualization, data analysis and closed-loop management is built. This map can not only provide data-based business management, help managers monitor and measure business indicators in real time, but also quickly locate the root cause of the problem by disassembling the indicators.

[0106] The indicator map needs to be describable, measurable and systematized. Describable means that each indicator should correspond to the actual business and reflect the specific affairs in the real business. Measurable means that business indicators should be digital, specific and measurable data. Systematized means that business indicators should form a systematic structure according to business processes or categories, and support the decomposition of indicators. Through this systematic indicator map, managers can quickly identify problems in the overall situation and analyze the root causes of the problems in depth by decomposing indicators.

[0107] The steps to construct an indicator map include:

[0108] Sort out the quality indicators in the system and clarify the business logic relationship between the indicators. In this step, first of all, it is necessary to sort out all the relevant quality indicators in detail and clarify the relationship and business logic between each indicator and other indicators. For example, the calculation of the yield rate indicator may be affected by multiple factors such as temperature and equipment status, and it is necessary to understand how each factor interacts with each other.

[0109] According to the indicator relationship, sort out the background data themes and data links. Once the relationship between the indicators is clear, it is necessary to sort out the data themes and data links in the system to ensure that the logic of data flow is clear and can support subsequent indicator analysis and visualization.

[0110] Establish a mind-map-style indicator map based on the indicator relationship. Finally, construct a mind-map-style indicator map based on the indicator relationship that has been sorted out, so that the relationship between various indicators is clear at a glance. Managers can use this map to intuitively understand the completion status of each indicator and their mutual influence.

[0111] Requirements for the indicator map include:

[0112] Each indicator needs to display the planned value (target value) and the actual completion value on the map to facilitate users to compare and check whether the predetermined goal has been achieved.

[0113] If the actual completion value is greater than or equal to the planned value, it means that the indicator has achieved or exceeded the expected target, and is displayed normally and marked in green. If you further click on the indicator, the user can jump to a more detailed report to view the detailed data of the indicator.

[0114] If the actual completion value is less than the planned value, it means that the indicator has not achieved the expected target and is highlighted in red. At this time, the user can drill down to view the detailed information of the unfinished indicator, and a dialog box will pop up to display the possible influencing factors. After further clicking, you can jump to the corresponding detailed report to help users analyze the specific reasons for the unfinished.

[0115] The target value of the indicator is set in a bottom-up way, that is, the current indicator performance is compared with the best practices or industry leading levels. The target setting can be based on the best performance of the department or the top X% of performance, or it can be benchmarked against the performance of industry leaders.

[0116] The yield rate index is gradually broken down into different levels of indicators according to the needs of business management and control, such as primary indicators, secondary indicators, tertiary indicators, and quaternary indicators. The indicator system is shown in Table 1, and the completion status of indicators at each level is intuitively displayed. Users can quickly view the completion status of each level through the map, and deeply analyze the data through the detailed drill-down function.

[0117] For example, if the yield rate of the automotive sheet galvanizing production line is unqualified on that day, the indicator map will mark the indicator in red. After clicking, the user can view the completion status of the yield rate of the specific variety, as well as the related head and tail cutting and edge cutting amounts. Through this information, the specific cause of the problem can be quickly analyzed to help make rectifications and optimizations.

[0118] Table 1 Index system table

[0119]

[0120]

[0121] Table 2 is an analysis of the yield rate and loss reduction index of each hot rolling process, which reveals the completion status of different processes between the planned value, baseline value and actual value, and reflects the state of savings or overspending through the loss reduction amount. For example, the actual yield rate of the first hot rolling process is 98.38%, which is slightly lower than the planned value of 98.48%, but better than the baseline value of 98.23%, indicating that the overall process operation is relatively stable; while the yield rate of the second hot rolling process is 98.16%, which is not only lower than the planned value and the baseline value, but also the loss reduction amount is -847,800 yuan, reflecting potential problems and the need for process improvement. Similarly, the actual yield rate and loss reduction overspending of the second acid trimming (-102,400 yuan) indicate that it needs more precise process optimization. In Table 2, the completion and benefit accounting of each unit index, + is better than the planned value, - is worse than the baseline, and the value in between is not marked. In actual production, the red mark is better than the planned value, the green mark is lower than the baseline value, and the black mark is between the baseline value and the planned value. The value 0 indicates that this type of steel is not produced in this statistical interval.

[0122] Table 2 Completion of indicators in query area of ​​each hot rolling process Table 3 shows the yield rate and loss reduction of each cold rolling process, and the data reveals the specific performance and optimization potential of different processes. For example, the actual yield rate of the pickling mill was 98.64%, higher than the planned value of 98.52%, and the loss reduction exceeded the target by 395,580 yuan, showing the benchmarking role of efficient production and loss reduction management; while the yield rate of the No. 2 galvanizing mill was 97.49%, which was close to the planned value, but the loss reduction was only 34,660 yuan, showing great optimization potential. Combined with the stability analysis of the continuous annealing and galvanizing processes, it is shown that these processes still need to be improved to achieve synchronous efficiency with pickling.

[0123] The above data can be processed through the source data layer and the basic subject layer to construct a multidimensional data mart for in-depth analysis. For the pickling mill with excellent performance, its control strategy and key indicators can be refined into a template through the data mart to support the imitation of other processes. For the continuous annealing and galvanizing units, combined with the yield rate indicator map and analysis results, the reasons for their substandard yield rate and loss reduction can be further explored, such as insufficient equipment maintenance, fluctuations in process parameters, etc. After reconstructing the logical calculation, accurately locating the problem links and optimizing the process flow will help the overall cold rolling process achieve a significant improvement in production efficiency and resource utilization, ensuring that the entire process meets the efficient and precise management goals proposed by the patent solution.

[0124] Table 3 Completion of indicators in query area of ​​each cold rolling process

[0125] Process Baseline % plan% Denominator molecular Yield rate % Monthly loss reduction plan Impairment amount / 10,000 yuan Pickling mill 98.36 98.52 95957.520 94650.020 98.64+ 24.98 39.558 Continuous retreat unit 97.69 97.84 44569.527 43656.890 97.95+ 39.43 28.411 Galvanizing Unit 1 97.11 97.41 26420.437 25690.730 97.24 33.68 11.950 Galvanizing Unit 2 97.45 97.60 23702.837 23108.230 97.49 32.28 3.466 Rewinding unit 98.43 98.58 29303.906 28838.420 98.41- 19.82 0.134

[0126] Table 4 is an analysis of the yield trend of cold rolling and the whole process of automobile plates, which reveals the cold rolling operation department and its performance in the overall production chain, and is directly related to the core value of the yield data analysis method proposed by the patent of this invention. The yield rate of the whole process of cold rolling represents the yield rate of the cold rolling operation department from the weight of the incoming materials (i.e., the materials supplied by hot rolling to cold rolling) to the weight of the cold-rolled finished products, that is, the final finished products / cold-rolled raw materials. The yield rate of the whole process represents the yield rate statistics from steelmaking slabs to the final automobile plate products, that is, the weight of automobile plates in cold rolling / the weight of steelmaking slabs. The yield rate of the whole process of cold rolling reflects the efficiency from the raw materials supplied by hot rolling to the final cold-rolled products, and the yield rate of the whole process further extends to the comprehensive performance from steelmaking slabs to the final automobile plate products.

[0127] Among them, No. 1 is steel grade CR180BH, No. 2 is steel grade CR290Y490T-DP, No. 3 is steel grade CR3, No. 4 is steel grade CR4, No. 5 is steel grade DC53D+Z, and No. 6 is steel grade DC54D+Z; Among them, "CR" in CR180BH steel indicates cold rolling, "180" indicates the minimum yield strength of 180MPa, and "BH" indicates bake hardening steel, which is often used in automobile manufacturing for parts that need to increase strength during the painting and baking process. In CR290Y490T-DP steel, "290Y" indicates the minimum yield strength of 290MPa, "490T" indicates the minimum tensile strength of 490MPa, and "DP" indicates dual-phase steel. Dual-phase steel has high strength and good ductility, and is suitable for parts that require high strength and formability, such as automotive structural parts. "3" in CR3 steel indicates the deep drawing quality grade. According to VDA 239-100, CR3 grade steel is suitable for applications that require good deep drawing properties, such as automotive body panels. The "4" in CR4 steel indicates an extra deep drawing quality grade. DC53D+Z, "DC" stands for cold rolled mild steel, "53D" for a specific quality grade, and "Z" for hot dip galvanized, has good formability and corrosion resistance. DC54D+Z is similar to DC53D+Z, but "DC54D" stands for a higher quality grade of cold rolled mild steel. DC54D+Z steel has even better formability and is suitable for applications that require higher formability.

[0128] Through the multidimensional data mart, the yield rate trend tracking of all stages of the whole process can be realized, and the efficiency bottlenecks from hot rolling supply to cold rolling completion can be clearly identified. Combined with the yield rate indicator map, the main reasons and key influencing chains for the change in the yield rate trend from January to November 2023 and 2024 can be intuitively displayed, such as the impact of a certain equipment failure in the cold rolling section on the ratio of hot rolling raw materials to finished products. At the same time, by comparing the actual and planned values, the links that are lower than expected can be quickly identified to optimize resource allocation and process coordination.

[0129] Ultimately, this application uses a full-chain analysis method from data acquisition, computational optimization to visual display, which can not only improve the efficiency of a single link (such as the cold rolling process), but also achieve comprehensive optimization of the entire process from steelmaking to automotive plates, providing a scientific basis for decision makers and helping to significantly improve production efficiency and yield rate.

[0130] Table 4 Full process trend of automotive sheet yield See also Figure 2 , is a schematic diagram of the structure of a yield rate data analysis device provided in an embodiment of the present application, comprising:

[0131] The yield rate data acquisition unit 21 is used to acquire the yield rate data of the corresponding process of each operation section, wherein the above-mentioned operation section includes the hot rolling operation section, the automobile sheet operation section and the silicon steel operation section;

[0132] The source data layer generation unit 22 is used to build an offline data warehouse based on the above-mentioned yield rate data using the big data platform, perform data preprocessing, and generate the source data layer;

[0133] A basic theme layer forming unit 23 is used to reconstruct the logic calculation process based on the above-mentioned post source data layer to form a basic theme layer;

[0134] The multi-dimensional data analysis unit 24 is used to construct a multi-dimensional data mart based on the basic subject layer, and perform multi-dimensional analysis on the yield rate data to obtain analysis results;

[0135] The indicator relationship analysis unit 25 is used to construct a yield rate indicator map based on the above analysis results to visualize the indicator relationship.

[0136] See also Figure 3 The embodiment of the present application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, the steps of any of the above-mentioned methods for analyzing yield rate data are implemented.

[0137] Since the electronic device introduced in this embodiment is the device used to implement the yield rate data analysis device in the embodiment of the present application, based on the method introduced in the embodiment of the present application, the technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is not introduced in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present application is within the scope of protection of this application.

[0138] During the specific implementation process, when the computer program 311 is executed by a processor, any implementation method in the embodiments corresponding to the first aspect can be implemented.

[0139] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-readable program code.

[0141] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0144] The present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device executes Figure 1 The process of the yield rate data analysis method in the corresponding embodiment.

[0145] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0147] In the several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0150] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0151] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0152] Although the preferred embodiments of this specification have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of this specification.

[0153] Obviously, those skilled in the art can make various changes and modifications to this specification without departing from the spirit and scope of this specification. Thus, if these modifications and variations of this specification fall within the scope of the claims of this specification and their equivalents, this specification is also intended to include these modifications and variations.

Claims

1. A method for analyzing yield rate data, characterized in that: Methods include: Obtaining the yield rate data of the corresponding process of each operation section, wherein the operation section includes a hot rolling operation section, an automobile sheet operation section, and a silicon steel operation section; Based on the yield rate data, an offline data warehouse is constructed using a big data platform to perform data preprocessing and generate a source data layer; Based on the post source data layer, reconstruct the logical calculation process to form a basic theme layer; Based on the basic theme layer, a multidimensional data mart is constructed to perform multidimensional analysis on the yield rate data to obtain analysis results; Based on the analysis results, a yield rate index map is constructed to visualize the index relationship.

2. The yield rate data analysis method according to claim 1, characterized in that: The data preprocessing step comprises: The yield rate data is extracted based on the offline data warehouse of the big data platform in an incremental manner, and outliers are eliminated and cleaned for the yield rate data.

3. The method for analyzing yield rate data according to claim 1, characterized in that: The offline data warehouse is a data warehouse supporting column storage built on Hadoop and Hive platforms.

4. The method for analyzing yield rate data according to claim 1, characterized in that: Also includes: The post source data layer is optimized using a Kudu database and / or a ClickHouse database based on column storage.

5. The method for analyzing yield rate data according to claim 1, characterized in that: The reconstruction logic calculation process specifically includes: Based on the Impala computing engine, a distributed computing method is used to perform logical calculation and index analysis on the yield rate data to obtain optimized yield rate data, wherein the logical calculation is achieved through hierarchical processing.

6. The method for analyzing yield rate data according to claim 1, characterized in that: The multidimensional data mart is constructed to perform multidimensional analysis on the yield rate data to obtain analysis results, including: Based on Kylin technology, pre-calculate the indicators after different dimensional combinations and store them in the data cube, wherein the different dimensional combinations are determined based on the yield rate data and the new dimensional data; A multi-dimensional analysis is performed based on the data cube to obtain an analysis result.

7. The method for analyzing yield rate data according to claim 1, characterized in that: Based on the analysis results, a yield rate index map is constructed to visualize the index relationship, including: Based on the analysis results, a relationship link between indicators is established to construct a yield rate indicator map; Marking the completion status of the indicator according to the indicator data of the yield rate indicator map, wherein the indicator data includes a planned value and an actual completion value; When the actual completion value is less than the planned value, the influencing factors are displayed.

8. A yield rate data analysis device, characterized in that: include: A yield rate data acquisition unit, used to acquire the yield rate data of the corresponding process of each operation section, wherein the operation section includes a hot rolling operation section, an automobile sheet operation section and a silicon steel operation section; A source data layer generation unit is used to build an offline data warehouse based on the yield rate data using a big data platform, perform data preprocessing, and generate a source data layer; A basic theme layer forming unit, used to reconstruct the logic calculation process based on the post source data layer to form a basic theme layer; A multi-dimensional data analysis unit, used to construct a multi-dimensional data mart based on the basic subject layer, perform multi-dimensional analysis on the yield rate data, and obtain analysis results; The indicator relationship analysis unit is used to construct a yield rate indicator map based on the analysis results to visualize the indicator relationship.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the yield rate data analysis method as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the yield rate data analysis method according to any one of claims 1 to 7 is implemented.

Images