Data processing methods, systems, and computer-readable storage media

By generating HBase tables and obtaining target detection data, the problem of insufficient production process records in industrial product manufacturing is solved, and efficient and accurate product defect analysis is achieved.

CN116724321BActive Publication Date: 2026-11-10BOE TECHNOLOGY GROUP CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202280000008.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-06
Publication Date
2026-11-10
Estimated Expiration
2042-01-06

AI Technical Summary

Technical Problem

In industrial product manufacturing, existing technologies are insufficient for effectively recording and analyzing the production process, resulting in inadequate timeliness and accuracy in locating the causes of product defects.

Method used

By generating an HBase table and using it as the target data table, target detection data matching the target detection parameters is obtained, and this data is displayed for data analysis. Production history and detection parameter data are stored using Hive tables and HBase tables, and the data is cleaned and transformed through the ETL module to generate an HBase table for data analysis.

Benefits of technology

It improves the efficiency and accuracy of product defect analysis, enabling the location and process of product defects to be identified, and reduces the difficulty of analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116724321B_ABST
    Figure CN116724321B_ABST
Patent Text Reader

Abstract

This disclosure relates to a data processing method, system, and computer-readable storage medium. The method includes: generating an HBase table based on a first Hive table and a second Hive table, and using the HBase table as a target data table; the first Hive table includes production history data for each product, and the second Hive table includes detection parameter data for products that have already been inspected; obtaining target detection parameters to be processed, wherein the target detection parameters are matched with the row keys of the target data table; obtaining target detection data corresponding to the target detection parameters based on the target detection parameter matching and the row keys of the target data table; the target detection data includes detection parameter data characterizing product defects; and displaying the target detection data to perform data defect analysis based on the target detection data. In this embodiment, matching the target parameter data with the product inspection scenario helps reduce the difficulty of data analysis and improves the efficiency and accuracy of analysis and inspection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a data processing method, system, and computer-readable storage medium. Background Technology

[0002] Currently, in the field of industrial product manufacturing, different products use different processes, are operated by different equipment and personnel, and even minor problems in any link can cause problems in the final industrial products. Therefore, it is necessary to record the production process of each industrial product to provide a basis for subsequent product defect analysis. Summary of the Invention

[0003] This disclosure provides a data processing method, system, and computer-readable storage medium to address the shortcomings of related technologies.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, comprising:

[0005] An HBase table is generated based on the first Hive table and the second Hive table, and the HBase table is used as the target data table; the first Hive table includes production history data of each product, and the second Hive table includes detection parameter data of products that have been tested.

[0006] Obtain the target detection parameters to be processed, wherein the target detection parameters are matched with the row key of the target data table;

[0007] The target detection data is obtained by matching the row key of the target data table with the target detection parameters; the target detection data includes detection parameter data characterizing product defects.

[0008] The target detection data is displayed so that data analysis can be performed based on the target detection data.

[0009] Optionally, an HBase table is generated based on the first Hive table and the second Hive table, including:

[0010] The production history information is extracted and stored in the first Hive table based on a preset first time interval; and the detection parameter data is extracted and stored in the second Hive table based on a preset second time interval.

[0011] The first Hive is converted into an HBase intermediate table with the product identification code as the row key;

[0012] The HBase table is obtained by merging the second Hive table and the HBase intermediate table.

[0013] Optionally, the HBase table is obtained by merging the second Hive table and the HBase intermediate table, including:

[0014] The main process station is obtained based on the detection data from the second Hive table;

[0015] Based on the main process site, the data in the HBase intermediate table is filtered to obtain the filtered HBase intermediate table;

[0016] The HBase table is obtained by merging the second Hive table and the filtered HBase intermediate table.

[0017] Optionally, obtaining the target detection data corresponding to the target detection parameters includes:

[0018] Based on the reference point of the reference product, the target detection data is obtained by averaging the detection data that are less than or equal to the reference point.

[0019] The benchmark product is the one with the most testing points.

[0020] Optionally, based on the reference points of the reference product, the target detection data is obtained by averaging the detection data at distances less than or equal to the reference points, including:

[0021] Match the control test points on the control product with the reference points on the reference product, respectively;

[0022] Obtain the distance between each control test point of the control product and the corresponding reference point on the reference product;

[0023] For each reference point on the reference product, acquire candidate detection data of the reference point and the control detection point when the distance to the control detection point is less than or equal to a preset distance threshold;

[0024] The average value of the candidate detection data is obtained, and a mean distribution map is generated using the average value of all detection points. The mean distribution map is then used as the target detection data.

[0025] Optionally, the process also includes a step of obtaining a benchmark product, specifically including:

[0026] Obtain the number of detection points for each product in the target data table, and identify the product with the most detection points. Use the product with the most detection points as the benchmark product, and use the other products as control products.

[0027] Optionally, the method further includes a step of post-processing the target detection data, specifically including:

[0028] Using each coordinate point in the mean distribution map as the center, a circle is obtained with a preset length as the radius, and the integer closest to the center of the circle is obtained;

[0029] Merge the coordinate points within the circle to obtain a single coordinate point corresponding to the circle; the coordinate data of the coordinate point is the nearest integer, which is used as the coordinate of the merged coordinate points within the circle.

[0030] Optionally, the target detection data further includes matching deviation data, and the method further includes:

[0031] Obtain the maximum and minimum values ​​of the product's detection data in the target detection data, as well as the average value corresponding to each detection point;

[0032] The deviation data for each detection point is calculated based on the maximum value, the minimum value, and the average value corresponding to each detection point.

[0033] Optionally, the method further includes:

[0034] The deviation data is visualized within the mean distribution graph.

[0035] Optionally, the target detection parameters are obtained through a preset filter, the filter having a download button, and the method further includes:

[0036] When an operation triggering the download button is detected, the target detection data is imported into a preset table;

[0037] The preset table is downloaded to a designated location so that users can perform data analysis based on the target detection data in the preset table.

[0038] According to a second aspect of the present disclosure, a data processing system is provided, comprising:

[0039] The target table acquisition module is used to generate an HBase table based on the first Hive table and the second Hive table, and use the HBase table as the target data table.

[0040] The target parameter acquisition module is used to acquire preset target detection parameters; the target data acquisition module is used to match the row key of the target data table according to the target detection parameters to obtain the target detection data corresponding to the target detection parameters; the target detection data includes detection parameter data characterizing product defects;

[0041] The target data display module is used to display the target detection data so that data analysis can be performed based on the target detection data.

[0042] According to a third aspect of the present disclosure, a data processing system is provided, including a data processing apparatus, the data processing apparatus comprising:

[0043] processor;

[0044] Memory for storing computer programs executable by the processor;

[0045] The processor is configured to execute a computer program in the memory to implement the method described above.

[0046] Optionally, it also includes a display device and a distributed storage device;

[0047] The distributed storage device is configured to acquire and store production history data and testing parameter data;

[0048] The display device is configured to display the target detection data.

[0049] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when an executable computer program in the storage medium is executed by a data processing apparatus, enables the implementation of the method described above.

[0050] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0051] As can be seen from the above embodiments, the solution provided in this disclosure can generate an HBase table based on a first Hive table and a second Hive table, and use the HBase table as the target data table; the first Hive table includes production history data of each product, and the second Hive table includes detection parameter data of products that have been inspected; the target detection parameters to be processed are obtained, and the target detection parameters are matched with the row keys of the target data table; the target detection data corresponding to the target detection parameters is obtained by matching the row keys of the target data table with the target detection parameters; the target detection data includes detection parameter data characterizing product defects; the target detection data is displayed to perform data analysis based on the target detection data. In this embodiment, the above-mentioned target parameter data can be selected based on the product process and historical experience, so that the target parameter data matches the product inspection scenario, which helps to reduce the difficulty of analyzing the defect process; and, by using the target detection data for defect analysis, the location and process information of product defects can be discovered, improving the efficiency and accuracy of analysis and inspection.

[0052] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0054] Figure 1 This is a block diagram illustrating a data processing system according to an exemplary embodiment.

[0055] Figure 2 This is a flowchart illustrating a data processing method according to an exemplary embodiment.

[0056] Figure 3 This is a flowchart illustrating the acquisition of a target data table according to an exemplary embodiment.

[0057] Figure 4 This is a schematic diagram illustrating the effect of a filter according to an exemplary embodiment.

[0058] Figure 5 This is a flowchart illustrating an example of acquiring target detection data according to an exemplary embodiment.

[0059] Figure 6 This is a flowchart illustrating an example of acquiring target detection data according to an exemplary embodiment.

[0060] Figure 7 This is a schematic diagram illustrating the effect of a mean distribution plot according to an exemplary embodiment.

[0061] Figure 8 This is a schematic diagram illustrating the effect of another mean distribution plot according to an exemplary embodiment.

[0062] Figure 9 This is a flowchart illustrating the merging of coordinate points according to an exemplary embodiment.

[0063] Figure 10 This is a flowchart illustrating the acquisition of deviation data according to an exemplary embodiment.

[0064] Figure 11 This is a block diagram illustrating a data processing apparatus according to an exemplary embodiment.

[0065] Figure 12 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described below by way of example do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatus consistent with some aspects of this disclosure as detailed in the appended claims. It should be noted that, without conflict, the following embodiments and features in the implementation methods can be combined with each other.

[0067] Currently, in the industrial product manufacturing sector, different products employ different processes, utilize different equipment, and are operated by different personnel. Even minor issues at any stage can lead to problems in the final industrial product. Therefore, it is necessary to record the production process of each industrial product to provide a basis for subsequent product defect analysis. Taking the semiconductor display industry as an example, in the production of display panels, the glass substrate undergoes different production processes and uses different equipment. Affected by objective factors such as each production process or equipment, the final display panel will produce various defects. In related technologies, technicians typically manually locate the causes of defects, making it difficult to meet production requirements in terms of processing time and accuracy.

[0068] To address the aforementioned technical problems, this disclosure provides a data processing method that can be applied to a data processing system. Figure 1 This is a block diagram illustrating a data processing system according to an exemplary embodiment. See also: Figure 1 The data processing system 100 includes a data processing device 300, a display device 200, and a distributed storage device 400. The data processing device 300 is connected to both the display device 200 and the distributed storage device 400. The distributed storage device 400 includes a data lake layer, a data warehouse layer (HIVE), and a data mart (HBASE). Users can input parameters to query through an interactive interface on the display device 200, and the display device 200 can also access the data mart via an API interface. The data processing device 300 can access the data mart via the API interface, thereby processing the data obtained from the data mart and sending it to the display device 200 for display.

[0069] See also Figure 1The data processing system includes multiple sets of data with different content and / or storage structures, stored within a distributed storage device 400. In some embodiments, an ETL module within the distributed storage device 400 can extract raw data from multiple data sources into the data processing system, forming a first data layer (e.g., a data lake layer, DL), to reduce the load on product manufacturing equipment and manufacturing systems, facilitating data retrieval by subsequent analysis equipment. The data sources can be raw data from manufacturing equipment, stored in relational databases (such as Oracle, MySQL, etc.) of corresponding manufacturing systems, such as YMS (Yield Management System), FDC (Fault Detection & Classification), and MES (Manufacturing Execution System). The aforementioned ETL module refers to computer program logic configured to provide functions such as extracting, transforming, or loading data. In some embodiments, the ETL module is stored on one or more storage nodes in a distributed network, loaded into one or more memories in the distributed network, and executed by one or more processors in the distributed network.

[0070] The data lake layer in the distributed storage device 400 is a centralized HDFS (Hadoop Distributed File System) or KUDU database used to store any structured or unstructured data. Optionally, the data lake is configured to store a first set of data extracted from multiple data sources (DS) by the ETL module. Optionally, the first set of data has the same content as the original data. The dimensions and attributes of the original data are preserved in the first set of data. In some embodiments, the first set of data stored in the data lake includes dynamically updated data. Optionally, the dynamically updated data includes data updated in real time by a Kudu-based database, or data updated periodically in the Hadoop Distributed File System. In one example, the periodically updated data stored in the Hadoop Distributed File System is stored in a Hive-based storage. In one example, the dynamically updated data also includes real-time updated data, where real-time updates are defined as updates below the minute level, excluding minute-level updates, to distinguish it from the periodic updates defined above as updates above the minute level and including minute-level updates.

[0071] In some embodiments, the distributed storage device 400 further includes a second data layer, such as a data warehouse. The data warehouse includes an internal storage system that provides data in an abstract manner without exposing a file system, wherein the abstraction may include tabular or view formats. The data warehouse may be implemented based on Hive. In this case, the ETL module can extract, clean, transform, or load the first set of data to form the second set of data. Optionally, the first set of data can be cleaned and standardized to form the second set of data. In some embodiments, the second set of data also includes statistical data, such as the number of detection points, the maximum, minimum, and average values ​​of detection point parameter values, and the defect percentage.

[0072] In some embodiments, the distributed storage device 400 includes a third data layer, such as at least one data mart. Optionally, the data mart is a NoSQL database that stores data that can be used for computational processing. Optionally, the data mart is implemented based on HBase. The ETL module can also transform the second data to form a third set of data.

[0073] Those skilled in the art will understand that the first set of data, the second set of data, and the third set of data can be stored and retrieved based on one or more data tables.

[0074] In some embodiments, the process of transforming the second set of data into the third set of data can involve importing data from a data warehouse (Hive tables) into a data mart (HBase tables). In one example, a first table is generated in the data mart, and a second table (e.g., an external table) is generated in the data warehouse. The first and second tables are configured to be synchronized so that when data is written to the second table, the first table is simultaneously updated to include the corresponding data. In another example, the MapReduce module in Hadoop can be used as a distributed computing processing module to read data written to the data warehouse. The data written to the data warehouse can then be written to the data mart. In one example, data can be written to the data mart using the HBase API. In another example, once the MapReduce module has read the data written to the data mart, it can generate HFile files and bulk-load them onto the data mart.

[0075] In some embodiments, data flow, data transformation, and data structure among various components of the data processing system are described. In some embodiments, raw data collected by multiple data sources DS includes at least one of production history data, parameter data, or detection parameter data. The raw data may optionally include dimensional information (time, plant, equipment, operator, map, chamber, slot, etc.) and attribute information (plant location, equipment age, number of bad pixels, abnormal parameters, energy consumption parameters, processing duration, etc.).

[0076] Production history data contains information about the specific processes a product (such as a panel or glass) undergoes during manufacturing. Examples of specific processes a product undergoes during manufacturing include the plant, process, site, equipment, chamber, slot, and operator.

[0077] Parameter data contains information about specific environmental parameters that a product (such as a panel or glass) experiences during manufacturing and how these parameters change. Examples of specific environmental parameters that a product experiences during manufacturing and how these parameters change include environmental particulate conditions, equipment temperature, and equipment pressure.

[0078] The test parameter data includes the resistance, film thickness, threshold voltage, reflective pattern offset, and reverse cutoff current of the product detected at the test site.

[0079] In one example, this data processing system integrates various business data (e.g., data related to semiconductor electronic device manufacturing) into multiple data sources (DS, e.g., an Oracle database). ETL modules, such as those using DataStack, SQOOP, Kettle, Pentaho, or DataX tools, extract data from these multiple data sources into a data lake. The data is then cleaned, transformed, and loaded into a data warehouse. The data warehouse (DW) and data mart (DMT) utilize tools such as Kudu, Hive, and HBase to store large amounts of data and analysis results.

[0080] Information generated at various stages of the manufacturing process is acquired by various sensors and inspection devices and subsequently stored in multiple data source databases (DS). Alternatively, calculations or analyses are performed on the data acquired by sensors and inspection devices, and the results are also stored in multiple data source databases (DS). Data synchronization (data flow) between various components of the data processing system is achieved through an ETL module. For example, the ETL module is configured to obtain a parameter configuration template for the synchronization process, including network permissions and database port configurations, inbound database names and table names, outbound database names and table names, field correspondences, task types, scheduling cycles, etc. The ETL module configures the parameters for the synchronization process based on the parameter configuration template. The ETL module synchronizes data and cleans the synchronized data based on the process configuration template. The ETL module uses SQL statements to clean the data, removing null values, removing outliers, and establishing correlations between related tables. Data synchronization tasks include data synchronization between multiple data sources and the distributed storage device 400, as well as data synchronization between different layers of the distributed storage device 400 (e.g., data lake, data warehouse, or data mart).

[0081] In another example, distributed storage device 400 can perform data extraction to the data lake either in real-time or offline. In offline mode, data extraction tasks are scheduled periodically. Optionally, in offline mode, the extracted data can be stored in a storage device based on the Hadoop Distributed File System (HDFS) (e.g., a Hive-based database). In real-time mode, data extraction tasks can be executed by OGG (Oracle GoldenGate) in conjunction with Apache Kafka. Optionally, in real-time mode, the extracted data can be stored in a Kudu-based database. OGG reads log files from multiple data sources (e.g., an Oracle database) to obtain data additions / deletions. In one example, a front-end interface (such as an API interface) can perform display, querying, and / or analysis based on data stored in a Kudu-based database. In another example, the front-end interface can be based on data stored in a Kudu-based database, a Hadoop Distributed File System (HDFS) (e.g., an Apache Hive-based database), and... T The ETL module is used to perform display, query, and / or analysis on data from any one or any combination of a Kudu-based database and / or an HBase-based database. In another example, short-term data (e.g., data generated over several months) is stored in a Kudu-based database, while long-term data (e.g., all data generated across all periods) is stored in a Hadoop Distributed File System (Hadoop Distributed File System), such as a Hive-based database. In yet another example, the ETL module is configured to extract data from a Kudu-based database into a Hadoop Distributed File System (Hadoop Distributed File System), such as a Hive-based database.

[0082] A data warehouse is built based on a data lake by combining data from various business systems (MDW, YMS, MES, FDC, etc.). Data extracted from the data lake is partitioned according to task execution time, which does not perfectly match the timestamps in the original data. Furthermore, there is a possibility of data duplication. Therefore, it is necessary to clean and standardize the data in the data lake to build the data warehouse, meeting the needs of upper-layer applications for data accuracy and partitioning. The data tables stored in the data warehouse are obtained by cleaning and standardizing the data in the data lake. Based on user requirements, field formats are standardized to ensure that the data tables in the data warehouse are completely consistent with data tables in multiple data sources. Simultaneously, partitioning data by date or month, time, and other fields greatly improves query efficiency and reduces runtime memory requirements. The data warehouse can be based on a Kudu-based database or an Apache Hive-based database, or any combination thereof.

[0083] In one embodiment, the distributed storage device 400 can be a single memory, multiple memories, or a collective term for multiple storage elements. For example, the memory may include: Random Access Memory (RAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SRAM), and may also include non-volatile memory, such as disk storage, flash memory, etc.

[0084] The display device 200 is used to display the interface and can display the processing results of the data processing device 300.

[0085] In one embodiment, the display device may be a monitor or a product that includes a monitor, such as a television, computer (all-in-one or desktop), tablet computer, mobile phone, electronic display screen, etc.

[0086] In one embodiment, the display device can be any device that displays images, whether moving (e.g., video) or stationary (e.g., still images), and whether it contains text or images. More specifically, the embodiments are contemplated to be implemented in or associated with a variety of electronic devices, such as (but not limited to) game consoles, television monitors, flat panel displays, computer monitors, automotive displays (e.g., odometer displays, etc.), navigators, cockpit controllers and / or displays, electronic photographs, electronic billboards or signs, projectors, architectural structures, packaging and aesthetic structures (e.g., a display of an image of a piece of jewelry), etc.

[0087] In one embodiment, the data processing apparatus 300 may be implemented using at least one server to implement the data processing method as described in any of the following embodiments, i.e., the data processing system implements the data processing method as described in any of the subsequent embodiments. Figure 2 This is a flowchart illustrating a data processing method according to an exemplary embodiment. See also Figure 2 A data processing method, comprising steps 21 to 24.

[0088] In step 21, an HBase table is generated based on the first Hive table and the second Hive table, and the HBase table is used as the target data table; the first Hive table includes production history data of each product, and the second Hive table includes detection parameter data of products that have been tested.

[0089] In this embodiment, the data processing system can obtain the target data table.

[0090] In one example, the target data table can be stored in a preset location, such as local storage, cache, or cloud. It is a target data table that has been pre-processed by other servers. In this scenario, the data processing system can read the target data table directly from the preset location and use it directly.

[0091] In another example, the target data table can be generated in real time by a data processing system, and this generation process can be synchronized with... Figure 1 The first, second, and third data layers of the distributed storage device 400 are matched. See also Figure 3 The data processing system can obtain the main process station based on the detection data of the second Hive table; then, based on the main process station, it filters the data in the HBase intermediate table to obtain the filtered HBase intermediate table; then, it merges the second Hive table and the filtered HBase intermediate table to obtain the HBase table.

[0092] For example, combining the target data table with production history data and testing parameter data for each product, the data processing system can communicate with the production system at a first time interval (e.g., each calendar day) and extract production history data (e.g., historical production history data and the production history data for the current day), that is, obtain the product's production history data (in an append-only manner). This loading process can be integrated with... Figure 1 The process of the distributed storage device 400 reading source data from the Oracle database into the data lake is matched. Then, the data processing system can extract, clean, transform, and load (ETL) the production history data (including historical production history data and current production history data) and store it in the first Hive table. The data structure of the first Hive table is shown in Table 1.

[0093] Table 1 GLASS Production History Data Table

[0094] field name significance Timeday Date of inspection Timekey Write time Factory factory glass_id Glass substrate identification code - Glass ID product_id Product ID step_id Process site eqp_id Process equipment unit_id Equipment unit

[0095] The data processing system can generate an HBase intermediate table based on the product production history data for the current day and / or historical production history data using the product identification code (such as Glass ID) as the row key in the first Hive table. The data structure of the HBase intermediate table is shown in Table 2.

[0096] Table 2. Structure Description of the Intermediate Table (MAP:Track)

[0097]

[0098] See also Figure 3 The data processing system can communicate with the testing system and load the testing parameter data from the testing parameter data table according to a second time interval (e.g., each natural day). Taking loading the testing data of the current day as an example, the data processing system can perform ETL processing on the product's testing parameter data and then store it in the second Hive table. The data structure of the second Hive table is shown in Table 3.

[0099] Table 3 (twyield.tw_glass_map) Detection Parameter Data Table

[0100]

[0101]

[0102] Then, the data processing system can read the daily detection data from the second Hive table. In some embodiments, outlier filtering can be performed on the parameters, where each parameter value corresponds to a range of values. This range can be set according to the product or based on empirical values, and is not limited here. In this example, filtering outlier parameters through this range can avoid the influence of outliers on subsequent calculation results, making the subsequent calculation results focus more on statistically significant trend values ​​rather than values ​​of special cases, which is more conducive to data analysis to locate the cause of the defects.

[0103] Furthermore, the data processing system can obtain a list of main process sites based on the daily detection parameter data in the second Hive table, or a list of main process sites uploaded by the user based on the product. In this example, obtaining the main process site list using the daily detection parameter data reduces the amount of data processing and improves the efficiency of obtaining the list. Moreover, obtaining the user-uploaded main process site list allows the data processing system to leverage the user's experience with the product manufacturing process and data analysis of issues such as positioning defects to obtain a more reliable site list. This leads to a more accurate list, which in turn improves the accuracy of subsequent processing results and enhances positioning efficiency.

[0104] The data processing system can obtain the intersection of the main process stations in the HBase intermediate table and the list of main process stations obtained based on the second Hive table, thus obtaining the main process stations for producing the product, as well as the corresponding test data for each main process station.

[0105] See also Figure 3 The data processing system can also process the filtered data, such as obtaining statistical information about the detection points, including the maximum, minimum, or average values ​​of the detection data. In this example, the large volume of production history data and detection parameter data can be used to obtain statistical information reflecting parameter trends, which can then be used to assist in locating defects.

[0106] See also Figure 3 The data processing system can aggregate the second Hive table, HBase intermediate points, the aforementioned main process stations, the detection data corresponding to the main process stations, and the statistical information of the detection points to obtain the aforementioned target data table. The data structure of the aforementioned target data table is shown in Table 4.

[0107]

[0108] Table 4 (MAP:Master) HBase Table Structure Description

[0109] In step 22, the target detection parameters to be processed are obtained, and the target detection parameters are matched with the row keys of the target data table.

[0110] In this embodiment, the data processing system can acquire target detection parameters. These target detection parameters include time range, factory, product model, detection site, and detection parameters; they may also include main process site, equipment, and unit. It should be noted that the first parameter in this embodiment includes time range, factory, product model, detection site, and detection parameters. The main process site, equipment, and unit are stored in the first designated column family (column family is Info, column name is MasterStep). Target detection parameters can be set according to requirements to avoid excessive data volume due to too many target detection parameters, thus improving query efficiency.

[0111] In one example, Figure 1 The display device 200 shown can display Figure 4 The filter shown allows users to select target detection parameters; that is, the target detection parameters can be selected through the filter. Figure 4 The filter shown is used to obtain the data. In this example, the filter can adopt a cascading structure, with the detection parameters of each level related to those of the levels above and below. After selecting the detection parameters of a certain level, all options of the detection parameters of the next level can be filtered out. In practical applications, the value of each item is selected by default as the first item under that option, but users can also arbitrarily select other option values ​​as filtering conditions. For example, in the first level, which is a time range, after detecting the start time (e.g., May 1, 2021) and end time (e.g., May 20, 2021), factories that participated in production within this time range can be filtered out (i.e., the second level); after detecting that the user has selected a factory (e.g., ARRAY), the model of the product already produced by that factory (e.g., BNA320WH5V401) can be filtered out (i.e., the third level). When it is detected that the user has entered the production batch / product ID under the model of the product being produced (i.e., the fourth level), and so on, the detection parameter to be queried (e.g., RS_DATA) can finally be obtained. Different detection parameters can represent different meanings, such as RS_DATA (surface resistance), THICKNESS (film thickness), TP (reflecting pattern offset), Vth (threshold voltage), and IOFF (reverse cutoff current). Understandably, technicians can select different parameters to create various target detection parameters based on their needs.

[0112] It should be noted that the time range in this example can be within one month (period). If the time interval between the start and end times exceeds one month, a reminder message will be generated, reminding the user that the entered time exceeds the time range and needs to be entered again. In this way, by setting a time range, this example avoids querying too much data each time, which helps improve query efficiency and ensures data validity.

[0113] It should be noted that in this example, multiple IoT IDs or Glass IDs can be entered in the input box for the IoT / Glass ID parameter. Adjacent IDs are separated by commas or spaces to achieve the effect of simultaneously detecting multiple parameters, thereby improving detection efficiency.

[0114] In step 23, target detection data corresponding to the target detection parameters is obtained based on the target detection parameter matching and the row key of the target data table; the target detection data includes detection parameter data characterizing product defects.

[0115] In this embodiment, the data processing system can obtain target detection data based on target detection parameters and a target data table. Based on the reference point of the benchmark product, it determines the target detection data by averaging the detection data at distances less than or equal to a set distance from the reference point. Specifically, this includes:

[0116] See one example. Figure 5 The data processing system acquires target detection data through steps 51 to 53.

[0117] In step 51, the data processing system can match the target detection parameters with the row keys in the target data table to obtain the original detection data corresponding to the target detection parameters.

[0118] Considering that the target detection parameters may include time range, factory, product model, testing site and testing parameters, in this step, each parameter can be matched with the row key in the target data table in turn to obtain the corresponding data; and so on, to obtain the original detection data that meets the above target detection parameters.

[0119] In step 52, the data processing system can acquire the detection point data of the column family in the original detection data to obtain the initial detection data corresponding to the target detection parameters. The data processing system can acquire the detection point data of each product under the column family (e.g., column family is Info, column name is MasterStep) in the original detection data to obtain the initial detection data. Here, the initial detection data refers to the collection of detection data from the detection points on (passed-inspection) products, where "passed-inspection" means that the detection system has already detected them.

[0120] In step 53, the data processing system can process the initial detection data to obtain target detection data.

[0121] It should be noted that the target detection parameters can also include parameters such as the main process station, equipment, and unit in the column family of the original detection data. In this case, the amount of initial detection data obtained will be much smaller than that of the target detection parameters. Figure 5The data volume of the illustrated scheme refers to the number of target detection parameters that a person skilled in the art can select according to the specific scenario to obtain initial detection data that meets the requirements. Alternatively, by increasing the number of parameters included in the target detection parameters, the final initial detection data can be reduced, which helps to reduce the data volume of subsequent processing and thus improves the efficiency of obtaining processing results.

[0122] In one embodiment, the data processing system can acquire a benchmark product, which can be a product pre-specified by the user or selected from past products. Considering the actual situation in the production process, the number of inspection points for some products may decrease, or the number of inspection points for some products may increase, meaning the number of inspection points for each product is random. Therefore, in this step, the data processing system can acquire the number of inspection points for each product, and then sort the number of inspection points for all products in the initial inspection data to obtain the product with the most inspection points. The data processing system can use the product with the most inspection points as the benchmark product, and the products other than the benchmark product as control products.

[0123] In this embodiment, see Figure 6 In step 53, the data processing system may acquire target detection data in steps 61 to 64.

[0124] In step 61, the data processing system can match the reference detection points on the reference product with the reference points on the reference product. Taking a glass substrate as an example, the data processing system can align the top corners or positioning marks of each reference product and the reference product, and then match the detection points on the reference product and the reference product sequentially. That is, the reference detection point in the first row and first column of the reference product is matched with the reference point in the first row and first column of the reference product, the reference detection point in the first row and second column of the reference product is matched with the reference point in the first row and second column of the reference product, and so on, until the detection point in the last row and last column is matched.

[0125] In step 62, the data processing system can obtain the distance between each control test point on the control product and the corresponding reference point on the reference product. The data processing system can use Euclidean distance to obtain the distance between two matching test points. For example, it can calculate the distance between the control test point in the first row and first column of the control product and the reference point in the first row and first column of the reference product, calculate the distance between the control test point in the first row and second column of the control product and the reference point in the first row and second column of the reference product, and so on, calculating the distance between the control test point in the last row and last column of the control product and the reference point in the last row and last column of the reference product.

[0126] In step 63, for each reference point on the reference product, the data processing system can acquire reference points and candidate detection data for control detection points whose distances are less than or equal to a preset distance threshold. The data processing system can store the preset distance threshold, which ranges from 1 to 10 mm and can be set according to specific scenarios. In one example, the preset distance threshold is 3 mm. The data processing system can compare the distance of each detection point with the preset distance threshold to obtain reference points and control detection points whose distances are less than or equal to the preset distance threshold, as well as the detection data for each detection point.

[0127] In step 64, the data processing system can obtain the average value of the detection data and generate a mean distribution map using the average value of all detection points, using the mean distribution map as the target detection data. In this step, for each detection point on the reference product, the data processing system can calculate the detection data of each detection point obtained in step 64, that is, the average value of the detection data of detection points (including this detection point on the reference product) whose distance from each detection point on the reference product is less than or equal to the aforementioned preset distance threshold.

[0128] In this step, the data processing system can generate a mean distribution map based on the coordinate data of each detection point on the benchmark product, combined with the average value of the corresponding detection data for that detection point. The effect is as follows: Figure 7 or Figure 8 As shown. It should be noted that, Figure 7 The mean distribution plot shown is based on Figure 5 The target detection data shown was generated. Figure 8 The mean distribution plot shown is in Figure 5 The target detection data shown is generated by adding parameters such as the main process station, equipment, and unit to the target detection parameters shown.

[0129] It should be noted that, Figure 7 and Figure 8 The first row of the mean distribution chart shows the x-coordinate of each detection point, the first column shows the y-coordinate of each detection point, and the other columns show the average value of the detection data for each detection point. A blank space indicates that there is no detection data at that coordinate.

[0130] It should be noted that, Figure 7 and Figure 8 The mean distribution chart shown is generated by pre-creating a table, then mapping the coordinate data of each detection point to the coordinate points in the table (i.e., the uniquely defined spaces for the horizontal and vertical axes), and importing the average value of each detection point into these coordinate points to obtain the final result. Figure 7 and Figure 8 The effect shown.

[0131] Understandably, in a pre-built table, adjacent horizontal or vertical coordinates can be set at equal intervals. In this case, the number of coordinate points (or cells) in the table will far exceed the number of test points (e.g., 70). Consequently, the mean distribution chart will contain many blank spaces, affecting its normal use. Therefore, the data processing system can further process the mean distribution chart as follows (see [link]). Figure 9 Specifically, this includes steps 91 and 92.

[0132] In step 91, the data processing system can form a circle with each coordinate point in the mean distribution map as its center and a preset length (e.g., 6 mm) as its radius, and obtain the integer closest to the center of the circle. In step 92, the data processing system can merge the coordinate points within the circle to obtain a single coordinate point corresponding to the circle; the coordinate data of the single coordinate point is the integer. Considering that the distance between each detection point (e.g., tens of mm) is usually greater than the preset length, forming a circle with the coordinate point as its center usually only includes that detection point. Therefore, the coordinate data of that detection point can be processed into an integer (closest to the center of the circle). Figure 9 The merging shown can significantly reduce the number of blank spaces in the mean distribution graph. Understandably, Figure 7 and Figure 8 This illustrates the effect of coordinate merging. After merging, the intervals between the x and y coordinates are no longer equal. For example, the difference between x-coordinate -1670 and x-coordinate -839 is 831 mm; the difference between x-coordinate -839 and x-coordinate -9 is 830 mm. Combined with reducing the intervals on the y coordinates, this significantly reduces the number of spaces, achieving... Figure 7 and Figure 8 The mean distribution graph shown achieves an effect where the number of coordinate points is similar to the number of detection points, ultimately resulting in the following: Figure 7 and Figure 8 The mean distribution plot shown facilitates the analysis of this mean distribution plot.

[0133] It should be noted that during outlier filtering, when multiple products show outliers at the same testing point, they need to be filtered out. This will also create blank spaces on the mean distribution chart. Therefore, the location of these blank spaces on the mean distribution chart is used to detect whether a product has a defect at the corresponding testing point. Because... Figure 9 The merging process shown in the example can greatly reduce the number of spaces, thus reducing the number of spaces that need to be checked, which helps improve the efficiency of bad analysis.

[0134] In one embodiment, considering that directly displaying numbers may not easily reveal defective locations, the target detection data in this example also includes matching deviation data, see [link to relevant documentation]. Figure 10 To obtain steps 101 and 102:

[0135] In step 101, the data processing system can obtain the maximum and minimum values ​​of the target detection data, as well as the average value corresponding to each detection point. For example, the maximum and minimum values ​​can be read from the structure shown in Table 4. In step 102, the data processing system can calculate the deviation data of each detection point based on the maximum and minimum values ​​and the average value corresponding to each detection point. The deviation data is used to assist in data analysis. The formula for calculating the deviation data is shown in equation (1) below:

[0136]

[0137] In equation (1), p i x represents the deviation at each detection point. min and x max The maximum and minimum values ​​of all test data on the product after matching the target test parameters from the target data table are respectively obtained, where x represents the test data of a certain test point.

[0138] In practical applications, the aforementioned deviation data can be simultaneously written into the aforementioned mean distribution chart, so that users can see the deviation data at the same time as they see the average value of each detection point, and use the average value and deviation data to analyze whether the detection point is a defective point, as well as other uncertain analyses.

[0139] In one embodiment, the data processing system can visualize the deviation data on a mean distribution chart. For example, the data processing system can convert the deviation data into a background color, and the intensity of the background color can represent the trend and degree of deviation of the product testing parameter mean. That is, the larger the deviation data, the darker the color, and the smaller the deviation data, the lighter the color. In one example, the intensity of the color can be set according to the rgba(0,255,0,p) and rgba(255,0,0,p) functions. Figure 8 The image shows the effect of using rgba(0,255,0,p) to represent bias data. Figure 9 The diagram illustrates the effect of using rgba(255,0,0,p) to represent deviation data. Those skilled in the art can choose an appropriate background color based on the specific scenario; considering the requirements of patent application documents, this disclosure uses grayscale representation to achieve the same effect.

[0140] In step 24, the target detection data is displayed for data analysis.

[0141] In this embodiment, the data processing system can send the aforementioned target detection data to a display device, which then displays the data, allowing the user to perform defect analysis based on the data. In other words, this embodiment can quantify the distribution trends of product detection parameters in different main production processes, thereby quickly identifying the causes of defects and enriching defect diagnosis and analysis methods.

[0142] In one embodiment, see further. Figure 4 The aforementioned filter can also be equipped with a download button (such as a similarity analysis button). When the filter detects that a user has triggered the download button, it can send a download request to the data processing system. Upon receiving the download request, the data processing system can import the target detection data into a preset table, which can be implemented using an Excel spreadsheet. Then, the data processing system can download the preset table to a designated location, allowing the user to perform data analysis based on the target detection data in the preset table.

[0143] Therefore, in this embodiment, the target parameter data can be selected based on the product manufacturing process and historical experience, ensuring that the target parameter data matches the product inspection scenario. This helps reduce the difficulty of analyzing defective processes. Furthermore, using target inspection data for defect analysis can reveal the location and process of product defects, improving the efficiency and accuracy of analysis and inspection. In other words, this embodiment, through multi-faceted analysis of product inspection parameters, provides analysis results with both theoretical basis and professional experience, helping users to more accurately and quickly pinpoint the causes of defects. Simultaneously, this embodiment is easy to operate, allowing users to control the data processing flow and thus improve production efficiency.

[0144] This disclosure also provides a data processing system, see [link to relevant documentation] Figure 11 ,include:

[0145] The target table acquisition module 111 is used to generate an HBase table based on the first Hive table and the second Hive table, and use the HBase table as the target data table.

[0146] Target parameter acquisition module 112 is used to acquire preset target detection parameters;

[0147] The target data acquisition module 113 is used to match the row key of the target data table according to the target detection parameters to obtain the target detection data corresponding to the target detection parameters; the target detection data includes detection parameter data characterizing product defects;

[0148] The target data display module 114 is used to display the target detection data so as to perform data analysis based on the target detection data.

[0149] It should be noted that the method shown in this embodiment is different from... Figure 1 The content of the method embodiments shown is consistent with that of the above method embodiments, and will not be repeated here.

[0150] In an exemplary embodiment, a data processing system is also provided, the data processing system including a data processing apparatus, see [link to relevant documentation]. Figure 12 The data processing device includes:

[0151] Processor 121;

[0152] Memory 122 for storing computer programs executable by the processor;

[0153] The processor is configured to execute a computer program in the memory to perform, for example, Figures 2 to 10 The method described.

[0154] In one embodiment, it further includes a display device and a distributed storage device;

[0155] The distributed storage device is configured to acquire and store production history data and testing parameter data;

[0156] The display device is configured to display the target detection data.

[0157] In an exemplary embodiment, a computer-readable storage medium is also provided, which, when an executable computer program in the storage medium is executed by a processor, enables the following: Figures 2 to 10 The method described.

[0158] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, magnetic disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drives (SSDs)).

[0159] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a processor, cause a computer to perform a data processing method as described in any of the above embodiments, such as one or more steps of the data processing method.

[0160] For example, the aforementioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards, and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices for storing information and / or other machine-readable storage media. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0161] The processor mentioned in the embodiments of this disclosure may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks and modules described in conjunction with this disclosure. The processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0162] Furthermore, the memory mentioned in the embodiments of this disclosure may be random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, portable hard disk, read-only optical disc (CD-ROM), or any other form of storage medium known in the art.

[0163] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0164] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized in that, include: An HBase table is generated based on the first Hive table and the second Hive table, and the HBase table is used as the target data table; the first Hive table includes production history data of each product, and the second Hive table includes detection parameter data of products that have been tested. Obtain the target detection parameters to be processed, wherein the target detection parameters are matched with the row keys of the target data table; The target detection data corresponding to the target detection parameters is obtained by matching the row key of the target data table with the target detection parameters. The target detection data includes detection parameter data characterizing product defects; Display the target detection data to perform data analysis based on the target detection data; Obtaining target detection data corresponding to the target detection parameters includes: acquiring a benchmark product; and determining, based on the benchmark points of the benchmark product, to obtain target detection data by averaging the detection data that are less than or equal to a set distance from the benchmark points. The steps for obtaining a benchmark product specifically include: obtaining the number of detection points for each product in the target data table, and obtaining the product with the most detection points; and using the product with the most detection points as the benchmark product, and using the products other than the benchmark product as control products. Based on the reference points of the benchmark product, the target detection data is obtained by averaging the detection data that are less than or equal to the reference points. Specifically, this includes: Match the control test points on the control product with the reference points on the reference product respectively; Obtain the distance between each control test point of the control product and the corresponding reference point on the reference product; For each reference point on the reference product, acquire candidate detection data of the reference point and the control detection point when the distance to the control detection point is less than or equal to a preset distance threshold; The average value of the candidate detection data is obtained, and a mean distribution map is generated using the average value of all detection points. The mean distribution map is then used as the target detection data.

2. The method according to claim 1, characterized in that, Generate HBase tables based on the first and second Hive tables, including: The production history information is extracted and stored in the first Hive table based on a preset first time interval; and the detection parameter data is extracted and stored in the second Hive table based on a preset second time interval. The first Hive is converted into an HBase intermediate table with the product identification code as the row key; The HBase table is obtained by merging the second Hive table and the HBase intermediate table.

3. The method according to claim 2, characterized in that, The HBase table is obtained by merging the second Hive table and the HBase intermediate table, including: The main process station is obtained based on the detection data from the second Hive table; Based on the main process site, the data in the HBase intermediate table is filtered to obtain the filtered HBase intermediate table; The HBase table is obtained by merging the second Hive table and the filtered HBase intermediate table.

4. The method according to claim 1, characterized in that, The method further includes a step of post-processing the target detection data, specifically including: Using each coordinate point in the mean distribution map as the center, a circle is obtained with a preset length as the radius, and the integer closest to the center of the circle is obtained; Merge the coordinate points within the circle to obtain a single coordinate point corresponding to the circle; the coordinate data of the single coordinate point is an integer.

5. The method according to claim 1, characterized in that, The target detection data also includes matching deviation data, and the method further includes: Obtain the maximum and minimum values ​​of the detection data in the target detection data, as well as the average value corresponding to each detection point; The deviation data for each detection point is calculated based on the maximum value, the minimum value, and the average value corresponding to each detection point.

6. The method according to claim 5, characterized in that, The method further includes: The deviation data is visualized within the mean distribution graph.

7. The method according to claim 1, characterized in that, The target detection parameters are obtained through a preset filter, which has a download button. The method further includes: When an operation triggering the download button is detected, the target detection data is imported into a preset table; The preset table is downloaded to a designated location so that users can perform data analysis based on the target detection data in the preset table.

8. A data processing system, characterized in that, include: The target table acquisition module is used to generate an HBase table based on the first Hive table and the second Hive table, and use the HBase table as the target data table; the first Hive table includes production history data of each product, and the second Hive table includes detection parameter data of products that have been tested. The target parameter acquisition module is used to acquire preset target detection parameters; The target data acquisition module is used to obtain target detection data corresponding to the target detection parameters by matching the row key of the target data table with the target detection parameters. This includes: acquiring a benchmark product; determining, based on the benchmark points of the benchmark product, to average the detection data obtained by averaging detection data that are less than or equal to a set distance from the benchmark points to obtain target detection data; the target detection data includes detection parameter data characterizing product defects; the step of acquiring the benchmark product specifically includes: acquiring the number of detection points for each product in the target data table, obtaining the product with the most detection points; and using the product with the most detection points as the benchmark product, and the products other than the benchmark product as control products; based on the benchmark... The target detection data is obtained by averaging the detection data at the reference points of the benchmark product, where the distance to the reference points is less than or equal to a set distance. Specifically, this includes: matching the reference detection points on the reference product with the reference points on the benchmark product; obtaining the distance between each reference detection point on the reference product and its corresponding reference point on the benchmark product; for each reference point on the benchmark product, obtaining candidate detection data for the reference points and the reference detection points when the distance to the reference detection points is less than or equal to a preset distance threshold; obtaining the average value of the candidate detection data; and generating a mean distribution map using the average value of all detection points, and using the mean distribution map as the target detection data. The target data display module is used to display the target detection data so that data analysis can be performed based on the target detection data.

9. A data processing system, comprising a data processing device, characterized in that, The data processing device includes: processor; Memory for storing computer programs executable by the processor; The processor is configured to execute a computer program in the memory to implement the method as described in any one of claims 1 to 7.

10. The data processing system according to claim 9, characterized in that, It also includes display devices and distributed storage devices; The distributed storage device is configured to acquire and store production history data and testing parameter data; The display device is configured to display the target detection data.

11. A computer-readable storage medium, characterized in that, When the executable computer program in the storage medium is executed by a processor, it can implement the method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Bad root cause positioning method based on product production records and parameters

    CN111159645A

  • Method and apparatus for detecting fault in the semiconductor menufacturing process and recording medium thereof

    KR101615346B1