Metal high-throughput experimental data automatic cleaning and tracing integrated management system

Abstract

The application relates to the technical field of data processing, in particular to a metal high-throughput experiment data automatic cleaning and tracing integrated management system, which establishes a sample point record by taking a batch number and a row and column number as a primary key; extracts a performance value and a space aggregation interval from a historical reserved point, generates a waveform template by bidirectional iteration constraint filtering and typical reserved subset generation; calculates waveform matching degrees of each sample point and the template, and determines a candidate set and waveform credibility by combining space density verification; solidifies a first round density reference system, and adaptively updates a judgment threshold value by driving the waveform credibility; calculates a local outlier factor score by fixing the reference system in each round, and compares the local outlier factor score with a threshold value to determine an abnormality, while recording a judgment process and a historical state throughout the process. The application can solve the semantic conflict between manual correction and algorithm rejudgment, reasonably adjust the judgment threshold value according to the waveform credibility, reduce the workload of manual review, trace the judgment process, and stabilize and ensure the cleaning result.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to an integrated management system for automatic cleaning and traceability of high-throughput metal experimental data. Background Technology

[0002] In high-throughput metal experiments, a sample library with varying composition and process parameters is prepared in parallel on the same sample disk. Adjacent wells exhibit continuous performance data due to their similar compositions. After testing, the data needs to be cleaned to remove outliers introduced by equipment drift, fabrication defects, etc. The current workflow first runs an outlier detection algorithm to output anomaly labels based on numerical distribution. Then, researchers verify and correct these labels point by point based on external information such as fabrication records, test waveforms, or equipment logs. Finally, a data table with retained or rejected labels is saved for subsequent modeling and analysis.

[0003] High-throughput experiments proceed in batch iterations. Some data has already been manually corrected and labeled in the previous round of cleaning based on external evidence such as waveform files. When recalculating the statistical distribution to update outlier determinations after adding new data, the following problems may occur: When manually reviewed data participates in the automated outlier detection algorithm's decision update, it can lead to an abnormal convergence direction of the algorithm's decision rule update; subsequently, the abnormally updated algorithm will generate a larger amount of misclassified data when processing the next batch of data, increasing the amount of manually reviewed data, which in turn affects the new round of algorithm decision updates, thus creating a vicious cycle. The root cause is that the physical basis on which manual judgments rely and the statistical judgments of the algorithm are not in the same semantic space—experimenters make decisions to retain or remove data based on waveform distortion and spatial position relationships, while outlier algorithms can only process numerical distributions and cannot understand physical semantics. Therefore, the problem to be solved is to structure these physical constraints and embed them into the outlier algorithm, so that the iterative calculation is compatible with both statistical distribution and physical rationality. Summary of the Invention

[0004] To address the aforementioned problems, this application provides an integrated management system for automatic cleaning and traceability of high-throughput metal experimental data, the system comprising: The data acquisition module is used to establish a data record for each sample point, which includes spatial coordinates, performance data and historical cleaning status, using the batch number and the row and column number of the sample point matrix as the primary key. The performance data includes performance test values ​​and original test waveforms, and the historical cleaning status is the retained label or the removed label after the previous round of manual verification. The historical pattern extraction module is used to extract the clustering intervals and spatial clustering regions of performance test values ​​from historical retention points. It uses bidirectional iterative constraints of the clustering intervals and spatial clustering regions of performance test values ​​to filter out typical retention subsets and extract waveform filtering templates from the waveforms of typical retention subsets. The waveform matching correction module is used to calculate the waveform matching degree between each sample point in this round and the waveform screening template, and to determine the candidate set and the corresponding waveform credibility based on the waveform matching degree and spatial density verification. The outlier detection update module is used to solidify the density reference system established in the first round of data and drive the adaptive update of the detection threshold with waveform credibility to generate the updated detection threshold. The iterative cleaning and source tracing module is used to calculate the local outlier score with the performance test value sequence of data in each round as input, with a fixed reference density reference system, and compare the local outlier score with the judgment threshold to identify outliers. In multiple iterations, it continuously performs outlier cleaning with the updated judgment threshold, and at the same time records the judgment process and historical cleaning status of each sample point.

[0005] The embodiments of this application have the following beneficial effects: This application provides an integrated management system for automatic cleaning and traceability of high-throughput metal experimental data. Sample point records are established using batch number and row / column number as primary keys, including spatial coordinates, performance values, original waveforms, and historical cleaning tags. Performance values ​​and spatial clustering intervals are extracted from historically retained points. Typical retained subsets are selected through bidirectional iterative constraints, and waveform templates are generated. The waveform matching degree between each point and the template is calculated, and the candidate set and its waveform credibility are determined by combining spatial density verification. The initial density reference system is solidified, and the judgment threshold is adaptively updated driven by waveform credibility. In each round, the local outlier factor score is calculated with a fixed reference system and compared with the threshold to determine anomalies. The entire judgment process and historical status are recorded. This application can resolve the semantic conflict between manual correction and algorithmic re-judgment, eliminate density reference system drift during iteration, reasonably adjust the judgment threshold according to waveform credibility, reduce the workload of manual review round by round, ensure traceability of all judgment processes, and provide stable and reliable cleaning results.

[0006] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0007] To more clearly illustrate the implementation schemes of this application, the accompanying drawings used in the implementation schemes will be briefly introduced below. It should be understood that the accompanying drawings only show some implementation schemes of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from the accompanying drawings without creative effort.

[0008] Figure 1 This is a block diagram illustrating an integrated management system for automatic cleaning and traceability of high-throughput metal experimental data, according to an exemplary embodiment. Figure 2 This is a block diagram illustrating a data acquisition module according to an exemplary embodiment; Figure 3 This is a block diagram illustrating a historical pattern extraction module according to an exemplary embodiment; Figure 4 This is a block diagram illustrating yet another historical pattern extraction module according to an exemplary embodiment; Figure 5 This is a block diagram illustrating a waveform matching correction module according to an exemplary embodiment; Figure 6 This is a block diagram illustrating an outlier detection and update module according to an exemplary embodiment; Figure 7 This is a block diagram illustrating an iterative cleaning and tracing module according to an exemplary embodiment. Detailed Implementation

[0009] The specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0010] It should be understood that the term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description.

[0011] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. The modifications of "one" and "multiple" mentioned in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless explicitly stated in the context, they should be understood as "one or more". In the description of this application, unless otherwise stated, "multiple" means two or more, and other quantifiers are similar; "at least one item", "one item or multiple items", or similar expressions refer to any combination of these items, including any combination of single or multiple items.

[0012] Although operations or steps are described in a specific order in the accompanying drawings in the embodiments of this application, this should not be construed as requiring these operations or steps to be performed in the specific order or serial order shown, or requiring all of the shown operations or steps to be performed to obtain the desired result. In the embodiments of this application, these operations or steps may be performed serially; they may be performed in parallel; or a portion of these operations or steps may be performed.

[0013] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information. It is understood that before using the technical solutions of the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained. The application will now be described in conjunction with specific embodiments.

[0014] Figure 1 This is a block diagram illustrating an integrated management system for automatic cleaning and traceability of high-throughput metal experimental data, according to an exemplary embodiment. Figure 1 As shown in the figure, this application provides an integrated management system for automatic cleaning and traceability of high-throughput metal experimental data, which may include the following modules: The data acquisition module is used to establish a data record for each sample point, which includes spatial coordinates, performance data and historical cleaning status, using the batch number and the row and column number of the sample point matrix as the primary key. The performance data includes performance test values ​​and original test waveforms, and the historical cleaning status is the retained label or the removed label after the previous round of manual verification. The historical pattern extraction module is used to extract the performance test value cluster intervals and spatial cluster regions from historical retention points, filter out typical retention subsets through bidirectional iterative constraints of the performance test value cluster intervals and spatial cluster regions, and extract waveform screening templates from the waveforms of the typical retention subsets. The waveform matching correction module is used to calculate the waveform matching degree between each sample point in this round and the waveform screening template, and to determine the candidate set and the corresponding waveform credibility based on the waveform matching degree and spatial density verification. The outlier determination and update module is used to solidify the density reference system established in the first round of data, and drive the adaptive update of the determination threshold with the waveform credibility to generate the updated determination threshold. The iterative cleaning and tracing module is used to take the performance test value sequence of each round of data as input, calculate the local outlier factor score with a fixed reference to the density reference system, compare the local outlier factor score with the judgment threshold to determine outliers, and continuously perform outlier cleaning with the updated judgment threshold in multiple iterations, while recording the judgment process and historical cleaning status of each sample point.

[0015] For example, this system first establishes a data foundation through the data acquisition module. For each batch of high-throughput experiments, the batch number of the sample point and its row and column position in the dot matrix are used as the unique primary key. The four types of information of each point—spatial coordinates, performance test values ​​and complete waveforms, and manual labels in historical cleaning—are integrated into a regular data record.

[0016] Building upon this foundation, the historical pattern extraction module takes over, its core task being to uncover the implicit physical rationality boundaries from points previously manually identified as retained. These retained points include both regular points at the center of numerical distribution and special points that appear numerically outliers but whose waveforms and spatial locations indicate physical rationality. This module iteratively constrains the numerical distribution intervals and spatial distribution regions, continuously narrowing the scope until a "typical retained subset" is selected. This subset passes both numerical and spatial tests, representing typical samples of physical rationality.

[0017] Then, the waveform matching and correction module comes into play. It extracts a waveform template representing the normal form from the waveforms of the typical retained subset, and then compares the waveforms of all test points in this round with this template one by one, calculating the waveform matching degree of each. For points with significantly low matching degrees, they are not directly judged as abnormal, but their spatial neighborhood is further examined—if most of their neighbors are historically retained points and their waveforms are consistent with each other, it means that although the waveform of this point differs from the template, it may be due to intrinsic changes in material composition, and it can still be included in the candidate set. Each point entering the candidate set is given a waveform confidence score between 0 and 1.

[0018] The outlier detection and update module is responsible for maintaining the stability of the statistical outlier detection algorithm. It fixes the numerical distribution structure of the first round of data as the reference benchmark for all subsequent iterations, prohibits the backflow of manually reviewed data from polluting this benchmark, and dynamically adjusts the judgment threshold using the waveform credibility calculated earlier. When the average waveform credibility of suspected outliers in a batch is high, it means that these points are physically reasonable, and the system will appropriately relax the judgment threshold to avoid accidental deletion; otherwise, the threshold will be tightened.

[0019] In the final iteration, the cleaning and tracing module calculates the outlier score of each point in each round by referring to the initially established numerical benchmark, and then compares it with the judgment threshold of the current round to draw a conclusion on whether it is abnormal. At the same time, it fully records how each point is judged in each round and what threshold is used, so that the entire cleaning process is traceable.

[0020] The system in this application embodiment enables statistical outlier determination and physical rationality to work collaboratively within the same framework. The density reference system established in the first round is permanently fixed to prevent iterative drift. The waveform credibility drives the adaptive adjustment of the threshold to ensure tolerance for physically reasonable outliers. After multiple rounds of iteration, the workload of manual review decreases round by round. The determination process for each data point is traceable, and the cleaning results are stable and reliable.

[0021] Figure 2 This is a block diagram illustrating a data acquisition module according to an exemplary embodiment. For example... Figure 2As shown, the data acquisition module may include the following modules: The integrity verification submodule is used to check the integrity of the primary key of each data record and exclude sample points with missing required fields from subsequent calculations. The spatial coordinate establishment submodule is used to directly read the spatial coordinates of each sample point on the substrate from the prepared mask layout; The performance data establishment submodule is used to obtain the performance test values ​​of sample points and the corresponding original test waveforms. The original test waveforms are load-displacement curves, which are composed of data points sampled at equal time intervals and include three continuous physical stages: loading segment, load holding segment, and unloading segment. The cleaning status establishment submodule is used to record the historical tags after the previous round of manual verification. The value corresponding to the retained tag is 0, and the value corresponding to the removed tag is 1. The historical cleaning status is allowed to be empty for points that are entered into the database for the first time and have no historical records. The incremental data access submodule is used to access the new sample point data of the new round of experiments. The new sample point data includes spatial coordinates and performance data, but does not include historical cleaning status.

[0022] For example, the data acquisition module takes on the responsibility of transforming raw experimental data into a structured, computable form from the start of system operation. It accomplishes this transformation through the collaborative efforts of five sub-modules.

[0023] The integrity verification submodule intervenes first to check whether the primary key of each record—that is, the batch number plus the row and column number—is complete. If a record is missing a batch number, row and column number, or performance test value or other required fields, the sample point is marked as "incomplete data" and isolated from subsequent calculations to prevent incomplete data from contaminating the analysis results.

[0024] The verified records are used to establish a spatial coordinate submodule. This submodule directly reads the spatial coordinates of each point from the mask layout file used during sample preparation. Since the mask layout defines the precise geometric arrangement of the dot matrix, the coordinates obtained not only accurately reflect the spatial position of each point, but more importantly, they can be used to calculate the topological adjacency relationship between any two points, providing a basis for subsequent spatial density verification.

[0025] The performance data establishment submodule obtains the performance test values ​​and complete waveforms for each point. Taking the nano-indentation test as an example, each point generates a complete curve of load changing with the indentation depth. This curve includes three continuous stages: loading segment, holding segment, and unloading segment. The equipment software extracts the hardness value from it, while the system saves the value and the original waveform at the same time. The value is used by the statistical outlier detection algorithm, and the waveform is used for subsequent waveform reliability evaluation.

[0026] The cleaning status establishment submodule records the results after the previous round of manual verification, with a retention flag of 0 and a rejection flag of 1. This field is empty for points entering the database for the first time. This provides a basis for the historical pattern extraction module to distinguish between retention points and rejection points.

[0027] The incremental data access submodule is specifically designed to handle newly added data points generated in the new round of experiments. The data structure of these data points is the same as described above, but they do not contain historical labels because they have not yet undergone any cleaning.

[0028] Through the collaboration of these five sub-modules, the system outputs a well-organized data table with primary keys as row identifiers. The spatial coordinates, performance data, and historical labels of each sample point are neatly stored in numerical form. This table becomes the input basis for all subsequent analysis modules, ensuring the consistency and traceability of data throughout the entire process.

[0029] Figure 3 This is a block diagram illustrating a historical pattern extraction module according to an exemplary embodiment. For example... Figure 3 As shown, the historical pattern extraction module may include the following modules: The initial performance test value candidate interval extraction submodule is used to sort the performance test values ​​of all historical retained sample points in ascending order, take the lower quartile of the difference between adjacent points as the window width, form a search interval with the minimum and maximum values ​​of the performance test values ​​of all historical tagged points, slide the window interval with the window width as the step size to obtain several window intervals, count the number of retained points and the number of rejected points in each window, calculate the ratio of the number of retained points to the number of rejected points, sort all the ratios of the windows in descending order, use the elbow point method to identify the inflection point as the first threshold, retain the window intervals with the ratio greater than the first threshold, and merge the performance test value range covered by the window intervals as the initial performance test value candidate intervals; The initial spatial candidate region extraction submodule is used to divide the sample point array into several grid cells with the average point distance as the grid side length, count the number of retained points and the number of rejected points in each grid cell, calculate the retention tendency, retain the grid cells with the retention tendency greater than the median of the retention tendency values ​​of all grid cells, and merge them into the initial spatial candidate region according to the eight-neighbor connectivity. The bidirectional iterative filtering submodule is used to calculate the first proportion of retained points in each window falling into the initial spatial candidate region, and to remove windows whose first proportion is lower than the median of the first proportion in all windows, and to merge the remaining windows to update the initial performance test value candidate interval. It also calculates the second proportion of retained points in each grid falling into the initial performance test value interval, removes grids whose second proportion is lower than the median of the second proportion in all grids, and reconnects and merges them to update the spatial candidate region. This process is repeated until the relative change of the performance test value interval boundary is less than 1% in two consecutive iterations and the change of the spatial region area is less than 5%, so as to obtain the final performance test value interval and the final spatial region. The typical retention subset output submodule is used to select all points from the historical retention points whose performance test values ​​fall within the range of the final performance test values ​​and whose spatial coordinates fall within the final spatial region, as the typical retention subset.

[0030] For example, the core of the historical pattern extraction module lies in identifying the statistical boundaries of physical rationality from past retained points, while the technical solution of this application embodiment solves this problem through the ingenious mechanism of "numerical-spatial bidirectional iterative constraint".

[0031] The initial performance test value candidate interval extraction submodule first works on the numerical dimension: it arranges the performance test values ​​of all historical retained points from smallest to largest, calculates the lower quartile of the difference between adjacent points as the window width, and then slides the window with this width from the minimum to the maximum value of all points, counts the number of retained points and rejected points in each window one by one, and calculates the ratio r of the retained point percentage.

[0032] For example, the ratio r can be obtained by the following formula: Formula 1 Where a is the number of points to keep in the window, b is the number of points to be removed in the window, and adding 1 is to avoid the denominator being zero. It can also be set to other non-zero minimum values, such as 0.001.

[0033] After sorting all windows by r value from high to low, the elbow point method is used to automatically identify the inflection point—that is, the position in the r value sequence where the rate of decrease suddenly slows down—as the threshold. Windows with values ​​higher than the threshold are retained and their coverage areas are merged to obtain the initial candidate interval of V value (performance test value).

[0034] Meanwhile, the initial spatial candidate region extraction submodule operates in the spatial dimension: it divides the grid using the average point spacing as the grid edge length, counts the number of retained and rejected points within each grid, calculates the retention tendency q, retains grids with q values ​​higher than the global median, and then merges adjacent high-tendency grids into initial spatial candidate regions according to the eight-neighbor connectivity rule. This yields two initial candidate regions, but the results of independent screening may include points that only meet numerical conditions but are spatially isolated, or points that only meet spatial conditions but have numerical anomalies.

[0035] For example, the retention bias q can be obtained by the following formula: Formula 2 Where c is the number of points retained in the grid cell, d is the number of points removed in the grid cell, and 1 is added to avoid the denominator being zero.

[0036] The bidirectional iterative filtering submodule calibrates these two ranges: first, it checks the proportion of retained points within each numerical window that fall within the current spatial region, removing windows with excessively low proportions and updating the V-value interval; then, it checks the proportion of retained points within each spatial grid that fall within the current V-value interval, removing grids with excessively low proportions and reconnecting and merging them to update the spatial region. This process iterates repeatedly until the change in the V-value interval boundary is less than 1% and the change in the spatial region area is less than 5%. The typical retained subset output submodule ultimately selects all historical retained points that fall simultaneously within the final V-value interval and the final spatial region as the typical retained subset.

[0037] This embodiment of the application uses bidirectional constraints to make the numerical statistical boundary and the spatial aggregation boundary mutually corroborate each other rather than simply superimpose them. It accurately extracts the most representative and physically reasonable samples from historical retention points, laying a solid foundation for the reliable generation of subsequent waveform templates.

[0038] Figure 4 This is a block diagram illustrating yet another historical pattern extraction module according to an exemplary embodiment. For example... Figure 4 As shown, the historical pattern extraction module further includes a waveform filtering template generation submodule, which may include the following units: The analysis segment extraction unit is used to calculate the first-order difference sequence for each load-displacement curve in the typical retained subset and assign signs according to positive and negative values. Taking the end position of the loading segment as the starting point of the load-preserving segment and the end position of the load-preserving segment as the starting point of the unloading segment, the starting segments of the load-preserving segment and the unloading segment are extracted and spliced ​​together to form an analysis segment, and linearly interpolated to a uniform number of sampling points. The starting segment of the unloading segment is the interval corresponding to one-quarter of the total length of the unloading segment, starting from the starting point of the unloading segment. The loading segment is the part of the first-order difference sequence that is positive, the load-preserving segment is the part of the first-order difference sequence that is zero, and the unloading segment is the part of the first-order difference sequence that is negative. The extreme value interval construction unit is used to divide the analysis segment with a unified number of sampling points into several sliding windows, each with a window length of one-third of the total length of the analysis segment and a step size of one sampling point, sliding from the beginning to the end of the analysis segment. Within each sliding window, the positions of the maximum and minimum load values ​​for each load-displacement curve are located, and the sub-segment between these values ​​is taken as the extreme value interval of the curve within the sliding window. If multiple identical maximum or minimum load values ​​appear within a sliding window, the position of the first occurrence in the sampling order is taken as the extreme value point. This is because the monotonic variation intervals within the loading and unloading segments typically do not produce multiple identical extreme values. Multiple equal extreme values ​​usually appear in the platform fluctuation region of the load-maintaining segment, where the curve shape changes gently, and taking the first occurrence position is sufficient to meet the extreme value interval positioning requirements.

[0039] The directional coding sequence generation unit is used to calculate the sign of the load difference between adjacent sampling points within the extreme value interval of the curve in each sliding window. The difference between the previous point and the subsequent point is recorded as +1 for positive values, -1 for negative values, and 0 for absolute values ​​of the change that are less than 1% of the load range of the load-displacement curve and less than 0.5% of the average load of the load-displacement curve, so as to obtain the directional coding sequence of the extreme value interval of the curve in the sliding window. The feature waveform segment filtering unit is used to encode all curves in each sliding window according to the direction of the extreme value interval, compare them pairwise with the same percentage interval, and calculate the proportion of the same sign bits to the total number of bits as the degree of coherence of the evolution trend of the two. The sum of the degree of coherence of the evolution trend of all curves is calculated, and the sliding window with the highest sum is taken as the feature waveform segment. The waveform screening template generation unit is used to take the median of the load values ​​of all curves corresponding to the percentage spacing sampling points in the typical retention subset in the characteristic waveform segment to generate a waveform screening template.

[0040] For example, the quality of the waveform screening template directly affects the reliability of subsequent waveform matching. The technical solution of this application specifically addresses the problem of how to extract representative morphological templates from numerous waveforms in a typical retained subset.

[0041] The analysis segment extraction unit first preprocesses each load-displacement curve: it calculates the first-order difference sequence and assigns positive and negative signs. The loading segment corresponds to a continuous interval with positive differences, the unloading segment corresponds to a continuous interval with negative differences, and the intermediate transition interval is the load-holding segment. Based on this, the complete interval of the load-holding segment and the starting segment (one-quarter of the distance from the start to the end point of the unloading segment) are extracted, spliced ​​together to form the analysis segment, and linearly interpolated to a uniform number of sampling points. This extraction strategy has a clear physical basis—the stability of the load-holding segment reflects whether the test conditions are stable, and the starting segment of the unloading segment is the direct source of performance value calculation. Together, they constitute the key interval for judging the reliability of the waveform.

[0042] The extreme value interval construction unit divides the analysis segment of uniform length into several windows with one-third of the total length as the window and one sampling point as the step size. For each curve in each window, the positions of the maximum and minimum loads are located, and the sub-segment between the two is taken as the extreme value interval of the curve in that window.

[0043] The directional coding sequence generation unit calculates the load change sign of adjacent sampling points for each of these extreme value intervals, recording positive values ​​as +1, negative values ​​as -1, and small changes (e.g., less than one percent) as 0, to obtain the directional coding sequence of each curve in each window. This sequence, in the form of a symbol string, describes the dominant rise and fall pattern of the curve in that interval.

[0044] The feature waveform segment selection unit solves the key problem: the criterion for selecting feature segments from numerous windows is the "degree of consistency of directional encoding". That is, the proportion of identical directional encoding between all curves in each window is calculated as the degree of synergy of evolutionary trends. Then, the sum of the synergy of the window is accumulated, and the window with the highest sum is selected as the feature waveform segment. This is because it means that the largest number of curves in the interval have the same dominant change trend, indicating that the window is in a common physical transition stage and is the most representative.

[0045] If the sum of the coordination degree of multiple windows is the same, the window whose center position is closest to the median of the analysis segment is taken as the characteristic waveform segment, so that the template truncation position is as close as possible to the intermediate physical transition stage shared by all curves.

[0046] The waveform screening template generation unit then takes the median of the load values ​​at the percentage interval sampling points of all curves in the typical retained subset within the characteristic waveform segment, synthesizing a template curve. The choice of the median, rather than the mean, makes the template insensitive to extreme waveforms and can stably reflect the common morphological characteristics of most curves.

[0047] The entire scheme of this application is based on the physical stage characteristics of the load-displacement curve, uses the synergy of the direction coding sequence as the screening criterion, and uses the point-by-point median as the synthesis method to ensure that the generated waveform screening template has high quality in both physical meaning and statistical robustness.

[0048] Figure 5 This is a block diagram illustrating a waveform matching correction module according to an exemplary embodiment. For example... Figure 5 As shown, the waveform matching correction module may include the following modules: The normalized difference value calculation submodule is used to extract the load value subsequence of the waveform screening template at any point in the current round, calculate the sum of the absolute values ​​of the point-by-point differences between the load value subsequence of the waveform screening template and the point-by-point differences between the point and the point, and then divide by the average load value of the waveform screening template within the window to obtain the normalized difference value between the point and the waveform screening template. The waveform matching degree calculation submodule is used to obtain the waveform matching degree based on the normalized difference value, the maximum value among the normalized difference values ​​of all points and the waveform screening template, and the minimum value among the normalized difference values ​​of all points and the waveform screening template. The natural boundary point identification submodule is used to sort all points according to the waveform matching degree from high to low, calculate the difference sequence of adjacent matching degrees, take the position corresponding to the maximum value in the difference sequence as the natural boundary point, and record the matching degree value of the point corresponding to the position as the first matching degree. If the maximum difference exceeds three times the second largest difference, the position of the second largest difference is taken as the natural dividing point. If the maximum difference is much greater than the second largest difference, it indicates that the jump may be isolated noise caused by a single anomaly rather than a distributional break at the population level. Taking the second largest difference can make the dividing point reflect a more stable population dividing position.

[0049] The candidate set is directly incorporated into the submodule, which is used to directly include points with waveform matching scores higher than the first matching score into the candidate set. The waveform confidence score corresponding to point p is... Waveform matching degree equal to the point ; The edge interval identification submodule is used to sort the points with waveform matching degree lower than or equal to the first matching degree again according to the matching degree from high to low, calculate the difference sequence of internal adjacent matching degrees, take the position of the maximum difference as the endpoint and record the matching degree of the endpoint as the second matching degree, and classify the points with matching degree between the first matching degree and the second matching degree as edge intervals. If the maximum difference appears in the last 10% of the sorted sequence, the position of the second largest difference is taken as the endpoint. The spatial density verification submodule is used to verify the location of each target point within the edge interval. Using a search radius of 1.5 times the average point spacing of the sample lattice, the set of all historically preserved points within its spatial neighborhood is delineated. The number of successful matches where the relative deviation between the target point and historically preserved points in the neighborhood is less than the global baseline deviation is counted. The ratio of the number of successful matches to the total number of points in the neighborhood is calculated as the local matching density. When the local matching density is greater than 0.5 and the total number of historical reserved points in the neighborhood is greater than or equal to 3, the target point is included in the candidate set; when the total number of historical reserved points in the neighborhood is less than 3, the target point is included in the candidate set only when the local matching density is equal to 1; when the number of historical reserved points in the neighborhood is zero, the local matching density is directly assigned to 0, and the above ratio calculation is no longer performed to determine the local matching density.

[0050] For example, the waveform matching correction module is responsible for comparing the waveform of each test point with the template and converting the comparison result into a confidence score that can participate in outlier determination. The solution of this application embodiment realizes this conversion through a three-layer progressive structure of "matching degree ranking - natural boundary identification - edge interval spatial verification".

[0051] The first step, the normalized difference calculation submodule, extracts a subsequence of load values ​​at each point within the same window position as the template, calculates the sum of the absolute differences at each point, and then divides it by the average load value of the template within that window to obtain the normalized difference value. The key to this step is to eliminate the baseline differences in load amplitude caused by the different material hardness at each point, so that the difference truly reflects the waveform shape rather than the difference in amplitude.

[0052] Based on this, the waveform matching degree calculation submodule maps the difference values ​​of all points to a matching degree range of 0 to 1, with smaller differences indicating higher matching degrees. The waveform matching degree is obtained based on the normalized difference value, the maximum value among the normalized difference values ​​of all points and the waveform filtering template, and the minimum value among the normalized difference values ​​of all points and the waveform filtering template.

[0053] Example: waveform matching degree at any point p It can be obtained from the following formula: Formula 3 in, Let p be the normalized difference between any point p and the waveform screening template. The maximum value among the normalized differences between all points and the waveform filtering template. It is the minimum value among all points and the normalized difference values ​​of the waveform screening template.

[0054] Formula 3-1 in, The normalized difference value between any point p and the waveform template; Let p be the load value at the a-th sampling point within the window; This represents the load value of the template at the a-th sampling point within the window. This represents the average load value of the template within the window; the denominator uses the average load of the template rather than the average load of the sample points to fix the normalization benchmark, making the differences between points comparable under the same reference system. A represents the total number of sampling points covered by the template feature segment. In actual calculations, if the total number of sampling points at a certain point is less than A or greater than A, the smaller value between the actual total number of sampling points at that point and A is taken as the number of sampling points to be included in the calculation.

[0055] The natural boundary point identification submodule sorts the waveform matching degree of all points from high to low, calculates the adjacent difference sequence, and takes the position with the largest difference as the natural boundary point—this position divides the sequence into two segments, with the largest difference between the segments and the smaller difference within the segments, indicating that there is a natural break in the matching degree distribution at this point. If the largest difference is more than three times the second largest difference, then the position of the second largest difference is taken as the natural boundary point.

[0056] The candidate set direct inclusion submodule directly includes high-matching points above the boundary point into the candidate set, and their waveform reliability is equal to the original matching degree. For points below the boundary point, the edge interval identification submodule recalculates the adjacent difference within it to identify an edge interval close to the boundary point—the matching degree of these points is lower than that of the boundary point, but the difference is not large, and the waveform matching degree alone cannot reliably determine their belonging.

[0057] The spatial density verification submodule initiates a spatial verification process for each point within the edge interval: it searches for historically preserved points in the spatial neighborhood with a radius of 1.5 times the average point distance, calculates the waveform matching degree between the point and each historically preserved point in the neighborhood, and obtains the relative deviation. This deviation is then compared with the global benchmark deviation, and the proportion of preserved points in the neighborhood with deviations lower than the benchmark is counted as the local matching density. When the local matching density is higher than 0.5 and there are at least 3 preserved points in the neighborhood, it indicates that although the matching degree of this point is at the edge, its waveform is consistent with the surrounding confirmed preserved points, possessing the physical rationale for spatial verification, and therefore it is included in the candidate set.

[0058] The hierarchical strategy of "initial matching + spatial verification" in this embodiment of the application makes the determination of the candidate set rely on both quantitative comparison of waveform morphology and group consistency verification of spatial neighborhood to prevent false exclusion caused by variations in the intrinsic waveform of the material. The two judgment dimensions complement each other.

[0059] Optionally, the waveform matching degree is a continuous value. The larger the value, the higher the consistency between the point and the waveform screening template, and the higher the physical reliability. The waveform confidence is a normalized value with a range of [0,1]. The waveform confidence of points with a waveform matching degree higher than the first matching degree is equal to their waveform matching degree. The waveform confidence of edge points included in the candidate set by spatial density verification is obtained based on the local matching density and the waveform matching degree. Points not included in the candidate set do not have waveform confidence.

[0060] For example, the embodiments of this application clearly define the connotation of the two core quantitative indicators, waveform matching degree and waveform credibility, and their specific application rules in the candidate set establishment process. This definition is the basis for the entire system to accept physical semantics at the numerical calculation level.

[0061] The waveform matching degree is defined as a continuous value, with a higher value indicating a greater consistency between the waveform at a given location and the waveform screening template. This continuous design allows for a finer characterization of the similarity between the waveform at each location and the template, rather than a coarse-grained binary classification, providing a numerical basis for subsequent natural boundary point identification and edge interval division.

[0062] During the process of determining the candidate set, points with waveform matching degree higher than the first matching degree are directly included in the candidate set. The waveform credibility of these points is equal to their original waveform matching degree - because their waveform shape is highly consistent with the template, and their physical credibility does not need to be discounted.

[0063] For target points whose matching degree is in the marginal range and require spatial density verification before being included in the candidate set... The target point For any sample point that requires spatial density verification before being included in the candidate set, its waveform reliability is... Based on local matching density Waveform matching degree It can be obtained through the following formula: Formula 4 The geometric meaning of this formula is that waveform confidence is the geometric mean of waveform matching degree and local matching density. If a point has a decent matching degree with the template but strong spatial neighborhood corroboration (ρ close to 1), the confidence is increased; conversely, if the spatial corroboration is weak, the confidence is suppressed. The geometric mean form ensures that if either factor is too low, the final confidence will be significantly reduced, avoiding the excessive influence of random values ​​in a single dimension on the judgment. Points not included in the candidate set do not have waveform confidence, meaning they have not passed the physical screening and will be subject to the numerical judgment of the statistical outlier algorithm.

[0064] This application embodiment uses this hierarchical assignment strategy to quantify the rationality of each candidate point in terms of physical morphology within the normalized range of 0 to 1, providing a calculable input for the outlier determination and update module.

[0065] Optionally, the global reference deviation is obtained in the following way: The relative deviation between a point within the edge interval and any historically preserved point in its neighborhood is obtained. The relative deviation is based on the waveform matching degree of the point within the edge interval and the waveform matching degree between the point within the edge interval and any historically preserved point in its neighborhood. The median of the relative deviation sequence between the point within the edge interval and all historically preserved points is taken as the global benchmark deviation.

[0066] For example, in the process of spatial density verification, a key parameter is the "global reference deviation". The embodiments of this application clarify the method of obtaining this parameter and establish a personalized reference for each edge point.

[0067] For a target point within the edge interval First, it is necessary to calculate the relative deviation between it and any historical preservation point Q in space. This relative deviation... It is not a simple difference in matching degree, but is obtained using the following formula: Formula 5 in, For target location Waveform matching degree between the waveform and the waveform template; For target location The waveform matching degree between the waveform and the historical reserved point Q can be referenced. The calculation method is determined; To obtain the maximum value.

[0068] The above formula normalizes the denominator by taking the larger of the two matching degrees, which is generally not zero, so that the deviation value always falls within the range of 0 to 1, eliminating the difference in deviation scale caused by different absolute values ​​of matching degree at different locations.

[0069] Get After considering a set of relative deviations from all historical retention points, the median of this set of values ​​is taken as... global reference deviation The median was chosen instead of the mean because... The waveform may deviate significantly from a few historically preserved points due to differences in material composition. These extreme values ​​can distort the mean, while the median's insensitivity to extreme values ​​allows it to stably reflect the mean. The "typical deviation level" from historical retention points on a global scale. The significance of this benchmark lies in: when examining... When considering the neighborhood, it is not required that every preserved point within the neighborhood is related to... Highly matched, but only if their matching deviation does not exceed The typical deviation level is sufficient for the global scope. In other words, it allows for gradual waveform differences due to variations in material composition gradients—if the waveform at a point matches the template poorly simply because its composition differs from the composition range represented by the typical retained subset, then its waveform should be consistent with the waveforms of retained points in the same composition range within its neighborhood, and the deviation of this consistency should not exceed its average deviation level globally.

[0070] This application's embodiments, by comparing neighborhood matching with a global benchmark, can effectively distinguish between two situations: "different waveforms due to material intrinsic differences" and "waveform distortion due to test anomalies," providing a physical and semantic basis for determining whether edge points can be included in the candidate set.

[0071] Figure 6 This is a block diagram illustrating an outlier detection and update module according to an exemplary embodiment. For example... Figure 6 As shown, the outlier detection and update module may include the following modules: The density reference system solidification submodule is used to establish the k-distance neighborhood structure of the local outlier factor algorithm based on the first round of data as the density reference system. All subsequent new batches will refer to the density reference system when calculating the local outlier factor score. The baseline threshold determination submodule is used to determine the baseline threshold based on the local outlier score distribution of the first-round dataset using an outlier detection method based on interquartile range. The average waveform confidence calculation submodule is used to calculate the average waveform confidence of all suspected outliers in the candidate set as the average waveform confidence for suspected outliers whose local outlier scores exceed the baseline threshold in the newly added batch. The threshold update submodule is used to obtain the updated judgment threshold based on the average waveform confidence level, the baseline threshold, and the preset tolerance coefficient.

[0072] For example, the outlier determination and update module is the key to breaking the vicious cycle of "manual verification of contaminated algorithm iteration" in this application. The technical solution of this application reconstructs the iterative mechanism of the traditional LOF (Local Outlier Factor) algorithm from two levels: "fixed density reference system" and "waveform credibility-driven threshold update".

[0073] Density reference system solidification submodule in the first round of data The k-distance neighborhood structure of the LOF algorithm is established as a density reference system. It also stipulates that all subsequent new batches will be calculated using a fixed reference when determining the LOF score. It is strictly forbidden to use data that has been manually reviewed. , The data is then merged back into the density calculation pool. The profound significance of this operation lies in the fact that the traditional process... and When merging and recalculating LOF, the points retained after manual verification (especially those numerically outliers but physically plausible) are re-included in the density calculation, which alters the global k-distance neighborhood structure, causing a shift in the density reference frame. Points that were originally outliers may no longer be outliers in the next round due to the change in the baseline, while points that were originally normal may be misjudged due to the change in the baseline. (Fixed) Subsequently, the numerical semantics of the LOF score remained consistent throughout the iterations, and the "outlier" was always relative to the numerical distribution in the first round, thus eliminating the source of contamination, baseline drift, to some extent.

[0074] The baseline threshold determination submodule uses the interquartile range method to determine the baseline threshold based on the LOF score distribution of the first-round dataset. This serves as the initial threshold for outlier detection. For newly added batches, the LOF score exceeds... The suspected outlier points are then identified, and the average waveform confidence calculation submodule calculates the average waveform confidence of these points within the candidate set. , A higher value indicates that although these suspected outliers are numerically outliers, their waveforms are well-formed and physically plausible. If the candidate set is empty, the average waveform confidence level is set to a fixed value of 0.

[0075] The threshold update submodule updates the judgment threshold according to the following formula and obtains the updated judgment threshold. .

[0076] Formula 6 Where λ is the preset tolerance coefficient, which can be 0.2 times. .

[0077] λ is used to bind the threshold adjustment range to the LOF score scale of this batch. If directly using... A fixed upward adjustment of 1 is insufficiently forgiving in batches with a wide LOF score distribution, and excessively forgiving in batches with a narrow distribution; λ is taken as... The 0.2 times that of the previous value allows the upward fluctuation to adapt to the data scale, avoiding the tolerance unevenness caused by a fixed absolute value.

[0078] It is worth noting that when the average waveform confidence of suspected outliers is higher, the judgment threshold can be appropriately raised, so that these physically reasonable but numerically outlier points have a chance to be leniently judged as normal; conversely, when the waveform confidence is low, the threshold remains basically unchanged, and real outliers will not be overlooked.

[0079] The entire scheme in this application, without touching the core position of manual review, ensures the stability of iterative numerical semantics by solidifying the density reference system, and injects physical semantics into the statistical judgment process by driving threshold updates through waveform credibility. The synergy of these two aspects ensures that the convergence direction of the outlier judgment criterion is stable and controllable.

[0080] Figure 7 This is a block diagram illustrating an iterative cleaning and tracing module according to an exemplary embodiment. For example... Figure 7 As shown, the iterative cleaning and tracing module may include the following modules: The local outlier factor score calculation submodule is used to calculate the local outlier factor score of each sample point with the performance test value sequence of each round of sample points as input and with fixed reference to the density reference system. The anomaly detection submodule is used to compare the local outlier score of each sample point with the current detection threshold. If the local outlier score is higher than the current detection threshold, it is judged as an anomaly point. If the local outlier score is lower than or equal to the current detection threshold, it is judged as a normal point. In the first round, the baseline threshold is used as the current detection threshold, and in subsequent rounds, the updated detection threshold is used as the current detection threshold. The judgment recording submodule is used to record the source of the judgment threshold, the local outlier score, and the waveform matching degree with the waveform screening template for each sample point, so as to form a traceable judgment process log. The historical status update submodule is used to associate and store the judgment results of each round of outlier cleaning with the final label after manual review, so as to update the historical cleaning status of each sample point.

[0081] For example, the iterative cleaning and traceability module is responsible for integrating the results of the preceding modules into a continuously running cleaning process and recording the judgment information throughout the process. The four sub-modules in this application embodiment constitute the execution framework of this process.

[0082] The local outlier factor score calculation submodule takes the original V-value (performance test value) sequence of sample points in each round as input, and fixes the density reference system established in the first round. Calculate the LOF score for each sample point. The key here is "fixed reference". No matter which round of iteration is performed, the LOF score of all points is calculated relative to the same k-distance neighborhood structure, which ensures that the scores between different rounds and batches are comparable and that the numerical semantics will not drift due to the addition of subsequent data.

[0083] The anomaly detection submodule compares the LOF score of each location with the current round's threshold; scores above the threshold are considered anomaly, while scores below or equal to the threshold are considered normal. The first round uses the interquartile range method to determine the score. As a threshold, subsequent rounds use the updated threshold obtained in the aforementioned embodiments. This design allows the judgment criteria to be adjusted appropriately based on the waveform's reliability, while remaining tolerant of physically reasonable outliers.

[0084] The judgment recording submodule establishes a detailed judgment log for each sample point, recording its LOF score, whether the threshold used for this judgment is a baseline threshold or an updated threshold, and its matching degree with the waveform screening template. After this information is fully saved, any point can be clearly shown why it was judged as abnormal or normal during subsequent tracing—whether it was because the outlier value exceeded the threshold, or because the waveform confidence drove the threshold to be raised and thus allowed to pass.

[0085] The historical status update submodule associates and stores the outlier cleanup results of each round with the final manually verified labels, updating the historical cleanup status of each sample point. This operation expands the set of historical retained and removed points after each iteration, providing a richer data foundation for the next round of historical pattern extraction. It is worth noting that if the automatic judgment result conflicts with the manually verified labels, the manually verified labels take precedence.

[0086] The four sub-modules in this application embodiment together constitute a closed-loop iterative system: the fixed reference LOF score ensures that the numerical judgment is stable and reliable, the waveform credibility-driven threshold update ensures that physical rationality is taken into consideration, the complete judgment record ensures that the process is traceable, and the continuous update of historical states ensures that the system can learn from each round of manual review.

[0087] An integrated management system for automatic cleaning and traceability of high-throughput metal experimental data may further include the following modules: The data table output module is used to output a regularized data table with the batch number and the row and column numbers of the sample point matrix as the row primary key. Each sample point in the regularized data table carries spatial coordinates, performance data, historical cleaning status labels and current cleaning judgment labels. The incremental iteration interface module is used to receive new sample point data from the new round of experiments, and after completing the outlier cleaning of this round, merge the judgment results of the new sample points with the historical batch data to form an updated and regularized data table for the next round of iteration.

[0088] For example, the data table output module and incremental iteration interface module of this application embodiment provide the entire system with standardized data output and the ability to continuously iterate.

[0089] The data output module outputs a well-organized data table with batch number and row / column number as the row primary key. Each sample point in this table carries all the key information from data acquisition to cleaning completion: spatial coordinates reflect the physical location of the point on the sample tray; performance data is the final test result extracted by the equipment software; the original test waveform is stored in the form of a complete load-displacement curve sequence, supporting subsequent secondary verification of cleaned data or extraction of new templates; historical cleaning status labels record the retention or rejection conclusions after the previous round of manual verification; and the current round of cleaning judgment labels record the judgment result automatically given by the system in the latest round of outlier cleaning. All information is output in a well-organized tabular format, allowing experimenters to clearly compare the consistency or difference between the system's automatic judgment and the manual verification results, facilitating the rapid identification of points requiring special attention.

[0090] The incremental iteration interface module is responsible for the system's continuous operation. When a new round of experiments generates new sample data, this module receives this data, whose structure is consistent with the original data but does not include historical cleansing states (because the new points have not undergone any manual verification). After completing this round of outlier cleaning, the incremental iteration interface module merges the judgment results of the new sample points with the historical batch data to form an updated, well-organized data table for the next round of iteration. This merging operation is not merely a simple data splicing; it also involves updating the data foundation of the historical pattern extraction module during the next round of operation. The retained and discarded points in the new batch, confirmed by manual review, will be included in the historical data pool, making the extraction of typical retained subsets increasingly accurate.

[0091] Through this mechanism, the system accumulates knowledge in each iteration, the waveform screening template becomes more and more representative as the number of retained points increases, the workload of manual review decreases round by round, and the accuracy and stability of the cleaning results are continuously improved.

[0092] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

Claims

1. An integrated management system for automatic cleaning and traceability of high-throughput metal experimental data, characterized in that, The system includes: The data acquisition module is used to establish a data record for each sample point, which includes spatial coordinates, performance data and historical cleaning status, using the batch number and the row and column number of the sample point matrix as the primary key. The performance data includes performance test values ​​and original test waveforms, and the historical cleaning status is the retained label or the removed label after the previous round of manual verification. The historical pattern extraction module is used to extract the clustering intervals and spatial clustering regions of performance test values ​​from historical retention points. It uses bidirectional iterative constraints of the clustering intervals and spatial clustering regions of performance test values ​​to filter out typical retention subsets and extract waveform filtering templates from the waveforms of typical retention subsets. The waveform matching correction module is used to calculate the waveform matching degree between each sample point in this round and the waveform screening template, and to determine the candidate set and the corresponding waveform credibility based on the waveform matching degree and spatial density verification. The outlier detection update module is used to solidify the density reference system established in the first round of data and drive the adaptive update of the detection threshold with waveform credibility to generate the updated detection threshold. The iterative cleaning and source tracing module is used to calculate the local outlier score with the performance test value sequence of data in each round as input, with a fixed reference density reference system, and compare the local outlier score with the judgment threshold to identify outliers. In multiple iterations, it continuously performs outlier cleaning with the updated judgment threshold, and at the same time records the judgment process and historical cleaning status of each sample point.

2. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 1, characterized in that, The data acquisition module includes: The integrity verification submodule is used to check the integrity of the primary key of each data record and exclude sample points with missing required fields from subsequent calculations. The spatial coordinate establishment submodule is used to directly read the spatial coordinates of each sample point on the substrate from the prepared mask layout; The performance data establishment submodule is used to obtain the performance test values ​​of sample points and the corresponding original test waveforms. The original test waveforms are load-displacement curves, which are composed of data points sampled at equal time intervals and include three continuous physical stages: loading segment, load holding segment, and unloading segment. The cleaning status establishment submodule is used to record the historical tags after the previous round of manual verification. The value corresponding to the retained tag is 0, and the value corresponding to the removed tag is 1. The historical cleaning status is allowed to be empty for points that are entered into the database for the first time and have no historical records. The incremental data access submodule is used to access the new sample point data of the new round of experiments. The new sample point data includes spatial coordinates and performance data, but does not include historical cleaning status.

3. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 1, characterized in that, The historical pattern extraction module includes: The initial performance test value candidate interval extraction submodule is used to sort the performance test values ​​of all historical retained sample points in ascending order, take the lower quartile of the difference between adjacent points as the window width, form a search interval with the minimum and maximum values ​​of the performance test values ​​of all historical tagged points, slide the window interval with the window width as the step size to obtain several window intervals, count the number of retained points and the number of rejected points in each window, calculate the ratio of the number of retained points to the number of rejected points, sort all the ratios of the windows in descending order, use the elbow point method to identify the inflection point as the first threshold, retain the window intervals with the ratio greater than the first threshold, and merge the performance test value range covered by the window intervals as the initial performance test value candidate intervals; The initial spatial candidate region extraction submodule is used to divide the sample point array into several grid cells with the average point distance as the grid side length, count the number of retained points and the number of rejected points in each grid cell, calculate the retention tendency, retain the grid cells with the retention tendency greater than the median of the retention tendency values ​​of all grid cells, and merge them into the initial spatial candidate region according to the eight-neighbor connectivity. The bidirectional iterative filtering submodule is used to calculate the first proportion of retained points in each window falling into the initial spatial candidate region, and to remove windows whose first proportion is lower than the median of the first proportion in all windows, and to merge the remaining windows to update the initial performance test value candidate interval. It also calculates the second proportion of retained points in each grid falling into the initial performance test value interval, removes grids whose second proportion is lower than the median of the second proportion in all grids, and reconnects and merges them to update the spatial candidate region. This process is repeated until the relative change of the performance test value interval boundary is less than 1% in two consecutive iterations and the change of the spatial region area is less than 5%, so as to obtain the final performance test value interval and the final spatial region. The typical retention subset output submodule is used to select all points from the historical retention points whose performance test values ​​fall within the range of the final performance test values ​​and whose spatial coordinates fall within the final spatial region, as the typical retention subset.

4. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 3, characterized in that, The historical pattern extraction module further includes a waveform filtering template generation submodule, which includes: The analysis segment extraction unit is used to calculate the first-order difference sequence for each load-displacement curve in the typical retained subset and assign signs according to positive and negative values. Taking the end position of the loading segment as the starting point of the load-preserving segment and the end position of the load-preserving segment as the starting point of the unloading segment, the starting segments of the load-preserving segment and the unloading segment are extracted and spliced ​​together to form an analysis segment, and linearly interpolated to a uniform number of sampling points. The starting segment of the unloading segment is the interval corresponding to one-quarter of the total length of the unloading segment, starting from the starting point of the unloading segment. The loading segment is the part of the first-order difference sequence that is positive, the load-preserving segment is the part of the first-order difference sequence that is zero, and the unloading segment is the part of the first-order difference sequence that is negative. The extreme value interval construction unit is used to divide the analysis segment after a unified number of sampling points into several sliding windows by sliding from the beginning to the end of the analysis segment with one-third of the total length of the analysis segment as the window length and one sampling point as the step size. For each load-displacement curve in each sliding window, the positions of the maximum and minimum load values ​​are located, and the sub-segment between the two is taken as the extreme value interval of the curve of the sliding window. The directional coding sequence generation unit is used to calculate the sign of the load difference between adjacent sampling points within the extreme value interval of the curve in each sliding window. The difference between the previous point and the subsequent point is recorded as +1 for positive values, -1 for negative values, and 0 for absolute values ​​of the change that are less than 1% of the load range of the load-displacement curve and less than 0.5% of the average load of the load-displacement curve, so as to obtain the directional coding sequence of the extreme value interval of the curve in the sliding window. The feature waveform segment filtering unit is used to encode all curves in each sliding window according to the direction of the extreme value interval, compare them pairwise with the same percentage interval, and calculate the proportion of the same sign bits to the total number of bits as the degree of coherence of the evolution trend of the two. The sum of the degree of coherence of the evolution trend of all curves is calculated, and the sliding window with the highest sum is taken as the feature waveform segment. The waveform screening template generation unit is used to take the median of the load values ​​of all curves corresponding to the percentage spacing sampling points in the typical retention subset in the characteristic waveform segment to generate a waveform screening template.

5. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 1, characterized in that, The waveform matching correction module includes: The normalized difference value calculation submodule is used to extract the load value subsequence of the waveform screening template at any point in the current round, calculate the sum of the absolute values ​​of the point-by-point differences between the load value subsequence of the waveform screening template and the point-by-point differences between the point and the point, and then divide by the average load value of the waveform screening template within the window to obtain the normalized difference value between the point and the waveform screening template. The waveform matching degree calculation submodule is used to obtain the waveform matching degree based on the normalized difference value, the maximum value among the normalized difference values ​​of all points and the waveform screening template, and the minimum value among the normalized difference values ​​of all points and the waveform screening template. The natural boundary point identification submodule is used to sort all points according to waveform matching degree from high to low, calculate the difference sequence of adjacent matching degrees, take the position corresponding to the maximum value in the difference sequence as the natural boundary point, and record the matching degree value of the point corresponding to the position as the first matching degree. If the maximum difference exceeds three times the second largest difference, the position of the second largest difference is taken as the natural boundary point. The candidate set is directly included in the sub-module, which is used to directly include points with waveform matching degree higher than the first matching degree into the candidate set. The waveform confidence degree corresponding to the point is equal to the waveform matching degree of the point. The edge interval identification submodule is used to sort the points with waveform matching degree lower than or equal to the first matching degree again according to the matching degree from high to low, calculate the difference sequence of internal adjacent matching degrees, take the position of the maximum difference as the endpoint and record the matching degree of the endpoint as the second matching degree, and classify the points with matching degree between the first matching degree and the second matching degree as edge intervals. If the maximum difference appears in the last 10% of the sorted sequence, the position of the second largest difference is taken as the endpoint. The spatial density verification submodule is used to, for each target point within the edge interval, use a search radius of 1.5 times the average point distance of the sample lattice as the search radius to delineate all historically preserved points in its spatial neighborhood. It then counts the number of successful matches where the relative deviation between the target point and the historically preserved points in the neighborhood is less than the global baseline deviation. The ratio of the number of successful matches to the total number of points in the neighborhood is calculated as the local matching density. When the local matching density is greater than 0.5 and the total number of historically preserved points in the neighborhood is greater than or equal to 3, the target point is included in the candidate set. When the total number of historically preserved points in the neighborhood is less than 3, the target point is included in the candidate set only when the local matching density is equal to 1.

6. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 5, characterized in that, The global reference deviation is obtained as follows: The relative deviation between a point within an edge interval and any historically preserved point in its neighborhood is obtained. The relative deviation is based on the waveform matching degree of the point within the edge interval and the waveform matching degree between the point within the edge interval and any historically preserved point in its neighborhood. The median of the relative deviation sequence between points within the edge interval and all historically preserved points is taken as the global baseline deviation.

7. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 6, characterized in that, The outlier detection and update module includes: The density reference system solidification submodule is used to establish the k-distance neighborhood structure of the local outlier factor algorithm based on the first round of data as the density reference system. All subsequent new batches will refer to the density reference system when calculating the local outlier factor score. The baseline threshold determination submodule is used to determine the baseline threshold based on the local outlier score distribution of the first-round dataset using an outlier detection method based on interquartile range. The average waveform confidence calculation submodule is used to calculate the average waveform confidence of all suspected outliers in the candidate set as the average waveform confidence for suspected outliers whose local outlier scores exceed the baseline threshold in the newly added batch. The threshold update submodule is used to obtain the updated judgment threshold based on the average waveform confidence level, the baseline threshold, and the preset tolerance coefficient.

8. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 7, characterized in that, The iterative cleaning and tracing module includes: The local outlier factor score calculation submodule is used to calculate the local outlier factor score of each sample point with the performance test value sequence of each round of sample points as input and with fixed reference to the density reference system. The anomaly detection submodule is used to compare the local outlier score of each sample point with the current detection threshold. If the local outlier score is higher than the current detection threshold, it is judged as an anomaly point. If the local outlier score is lower than or equal to the current detection threshold, it is judged as a normal point. In the first round, the baseline threshold is used as the current detection threshold, and in subsequent rounds, the updated detection threshold is used as the current detection threshold. The judgment recording submodule is used to record the source of the judgment threshold, the local outlier score, and the waveform matching degree with the waveform screening template for each sample point, so as to form a traceable judgment process log. The historical status update submodule is used to associate and store the judgment results of each round of outlier cleaning with the final label after manual review, so as to update the historical cleaning status of each sample point.

9. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 5, characterized in that, The waveform confidence is a normalized value with a range of [0,1]. The waveform confidence of points with a waveform matching degree higher than the first matching degree is equal to their waveform matching degree. The waveform confidence of edge points included in the candidate set by spatial density verification is obtained based on the local matching density and the waveform matching degree. Points not included in the candidate set do not have waveform confidence.

10. The integrated management system for automatic cleaning and traceability of high-throughput metal experimental data according to claim 1, characterized in that, The system also includes: The data table output module is used to output a regularized data table with the batch number and the row and column numbers of the sample point matrix as the row primary key. Each sample point in the regularized data table carries spatial coordinates, performance data, historical cleaning status labels and current cleaning judgment labels. The incremental iteration interface module is used to receive new sample point data from the new round of experiments, and after completing the outlier cleaning of this round, merge the judgment results of the new sample points with the historical batch data to form an updated and regularized data table for the next round of iteration.

Images