Machine data management method and device and storage medium

Abstract

The invention relates to the field of semiconductors, in particular to a machine data management method and device and a storage medium. The method comprises the following steps: acquiring a target process step executed by a target machine and machine data generated when the process step is executed; according to the process category and / or the process sequence of the target process step, time efficiency information of the machine data is determined; dividing a future time axis into a plurality of continuous and non-overlapping time intervals based on the starting and ending time of a window in the time efficiency information; calculating a comprehensive access probability of each time interval based on the predicted access probabilities corresponding to different pieces of time efficiency information, and allocating a corresponding compression strategy to each time interval based on the comprehensive access probability; and performing compression storage on the machine data by applying corresponding compression strategies in different time intervals. Therefore, the dynamic and on-demand distribution of the storage resources is realized, and the balance between the data storage cost and the response efficiency is achieved.

Description

Technical Field

[0001] This application relates to the semiconductor field, and more particularly to a method, apparatus, and storage medium for managing machine data. Background Technology

[0002] In the semiconductor manufacturing industry, as process nodes continue to evolve towards advanced processes, the complexity of chip production increases exponentially. Modern wafer fabs involve hundreds or even thousands of process steps, covering multiple key stages such as photolithography, etching, thin film deposition, ion implantation, and chemical mechanical polishing. These processes are typically completed collaboratively by a wide variety of specialized machines with diverse functions. To ensure yield and achieve precise process control, each machine integrates numerous high-precision sensors to monitor hundreds or even thousands of process parameters in real time, including temperature, pressure, gas flow rate, RF power, and vibration frequency.

[0003] To achieve refined monitoring, fault prediction, and root cause analysis of the production process, the industry tends to adopt high-frequency (millisecond or even microsecond-level sampling) and large dynamic range data acquisition strategies. While this strategy can capture transient process fluctuations and subtle anomalies, it also leads to an explosive growth in data volume. A single advanced machine operating at full capacity can generate tens of gigabytes or even terabytes of time-series data per day, and the total amount of data generated daily by an entire wafer fab can reach petabytes. Therefore, efficient data storage and management optimization are necessary.

[0004] For example, patent application CN119937937A discloses an intelligent storage method for data acquired by semiconductor devices, comprising: segmenting a time-series data sequence; determining the initial degree of anomaly of the data in each time period based on the trend anomaly characteristics of the data change trend in the time-series data sequence and the anomaly characteristics of the data cycle in each time period; determining the precise degree of anomaly of the data in each time period based on the probability of the existence of truly abnormal data in each time period and the initial degree of anomaly of the data in each time period; determining the dead zone threshold and dead zone range corresponding to each time period based on the precise degree of anomaly of the data in each time period and the mean of the absolute values ​​of the differences between adjacent data in each time period; and performing efficient compression processing on the time-series data sequence based on the dead zone range corresponding to each time period.

[0005] For example, patent application CN120811397A discloses a method, apparatus, electronic device, and medium for compressing semiconductor industry operation and maintenance data, relating to the field of data compression technology. The method includes: acquiring semiconductor industry operation and maintenance data to be compressed; identifying the temporal nature of the data and determining whether it is time-series data based on the temporal nature; if the data is time-series data, splitting it into time data groups and feature data groups, and compressing the time data groups and feature data groups separately.

[0006] However, while existing data storage and compression technologies have reduced data volume to some extent, they are insufficient to meet the stringent requirements for data fidelity and response speed in semiconductor manufacturing scenarios. Summary of the Invention

[0007] The main objective of this application is to provide a method, device, and storage medium for managing machine data. To solve the aforementioned technical problems, this application specifically adopts the following technical solution: A first aspect of this application is to provide a method for managing machine data, the method comprising: S101, Obtain the target process step executed by the target machine, and the machine data generated during the execution of this process step; S102, determine the timeliness information of the machine data according to the process category and / or process sequence of the target process step, wherein the timeliness information includes at least one of the following: life cycle stage window, analysis and evaluation window, and preset call window; S103, based on the start and end times of the window in the timeliness information, the future timeline is divided into several continuous and non-overlapping time intervals; S104, based on the predicted access probabilities corresponding to different time-sensitivity information, calculate the comprehensive access probability for each time interval, and assign a corresponding compression strategy to each time interval based on the comprehensive access probability; wherein, the compression degree of the compression strategy on the machine data is negatively correlated with the comprehensive access probability. S105 applies corresponding compression strategies to compress and store machine data in different time intervals.

[0008] In some embodiments, S104 includes: querying a preset data lifecycle stage table to determine a first prediction probability for a time interval; and / or obtaining a second prediction probability for a time interval within the analysis and evaluation window; and / or obtaining a third prediction probability for a time interval within the preset call window; when at least two probability components of the first, second, and third prediction probabilities are obtained, the at least two probability components are weighted and fused to obtain a comprehensive access probability; when any one of the first, second, and third prediction probabilities is obtained, the corresponding probability component is used as the comprehensive access probability.

[0009] In some embodiments, S104 includes: pre-establishing a mapping relationship between the overall access probability and the compression strategy; the mapping relationship includes: a preset number of levels, a probability threshold range corresponding to each level, and a compression strategy associated with each probability threshold range.

[0010] In some embodiments, S104 includes: adopting a first compression strategy when the overall access probability of a time interval is higher than a first probability threshold; adopting a second compression strategy when the overall access probability of a time interval is lower than or equal to the first probability threshold and higher than a second probability threshold; and adopting a third compression strategy when the overall access probability of a time interval is lower than or equal to the second probability threshold; wherein the compression degree of the first compression strategy is lower than that of the second compression strategy, and the compression degree of the second compression strategy is lower than that of the third compression strategy.

[0011] In some embodiments, the method further includes: acquiring a target machine group, the target machine group including a target machine and several other machines of the same type as the target machine, the other machines including prior machines with a service life longer than the target machine; determining the peak time period when the target machine data will be accessed in the future based on the data access logs of the prior machines when they reached the same service life as the target machine, and using the peak time period as a preset call window.

[0012] In some embodiments, the method further includes: acquiring in real time the measured access frequencies of multiple machines in the target machine group within a preset time period; calculating the group access frequency based on the measured access frequencies of the multiple machines; updating the compression strategy for the current time interval based on the group access frequency when abnormal fluctuations in the group access frequency are detected; and / or updating the compression strategy for the current time interval based on the measured access frequencies of the target machines.

[0013] In some embodiments, the method further includes: before performing compressed storage, updating the compression strategy, which includes: predicting the cumulative memory usage at different time points in the future time axis based on the compression strategy of multiple batches of machine data within the target machine; when the cumulative memory usage at any time point is greater than a preset memory threshold, updating the mapping relationship between the comprehensive access probability and the compression strategy at the corresponding abnormal time point, including: reducing the number of levels in the mapping relationship; and updating the compression strategy of the time interval where the abnormal time point is located based on the updated mapping relationship.

[0014] In some embodiments, the updated mapping relationship includes: when the overall access probability of a time interval is higher than a first probability threshold, a first compression strategy is adopted; when the overall access probability of a time interval is lower than or equal to the first probability threshold, a third compression strategy is adopted.

[0015] A second aspect of this application is to provide a computer device, the device comprising: Memory, used to store computer programs; A processor is configured to execute the computer program and, in executing the computer program, implement the steps of the machine data management method provided in any embodiment of this application.

[0016] A third aspect of this application is that a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the machine data management method provided in any embodiment of this application.

[0017] Beneficial technical effects: This application provides a method, device, and storage medium for managing machine data, specifically proposing a dynamic data recompression mechanism based on timeliness prediction and memory adaptation. By leveraging the technological characteristics of the machine data, compression strategies covering different time intervals throughout its entire lifecycle are planned. As the data lifecycle naturally evolves, different compression strategies are automatically invoked and recompression operations are performed, thereby achieving dynamic, on-demand allocation of storage resources and achieving a balance between data storage cost and response efficiency.

[0018] First, based on the process category of the technological steps performed by the machine and / or its process sequence in the production line flow, the key application periods of this type of machine data in subsequent manufacturing and analysis processes are identified from a multi-dimensional business perspective. Then, based on these application periods, the future timeline from the current moment is divided, generating several continuous and non-overlapping time intervals, each corresponding to a specific combination of time-sensitive information.

[0019] Based on this, predictive access probability components with different timeliness information are constructed and integrated to accurately predict the access frequency of machine data in future time periods, and accordingly, differentiated compression strategies are matched for each time interval: low compression is used in high access frequency intervals to ensure response efficiency, while high compression is enabled in low access frequency intervals to save storage costs, thereby achieving the adaptation of storage resource configuration to business needs.

[0020] Furthermore, a closed-loop control mechanism is established for the compression strategy from both the business demand side (i.e., response efficiency) and the resource supply side (i.e., storage cost). On the one hand, by combining the measured access frequency of the target machine itself and its group, the predicted access probability is calibrated in real time, effectively capturing sudden analysis needs or common process anomalies, improving the adaptability of the compression strategy, and enhancing response efficiency. On the other hand, the compression strategy for machine data throughout its entire lifecycle is pre-planned and predictable, supporting the estimation of future cumulative memory usage trends, thereby identifying unexpected increases in storage costs in advance and optimizing the compression strategy mapping relationship (such as reducing the number of levels and increasing the overall compression intensity) to address these issues.

[0021] This approach avoids both the inability to access critical data in a timely manner due to excessive compression and the waste of resources caused by retaining too much low-value hot data, thereby achieving refined management of storage costs and improving resource utilization. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of this application; for those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0023] Figure 1 This is a schematic diagram of a future timeline provided in an embodiment of this application; Figure 2 This is a schematic flowchart illustrating a method for managing machine data provided in an embodiment of this application; Figure 3 This is a schematic diagram of a comprehensive access probability generation architecture provided in an embodiment of this application; Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] In this document, suffixes such as “module,” “part,” or “unit” used to denote elements are used only for illustrative purposes and have no specific meaning in themselves. Therefore, “module,” “part,” or “unit” may be used interchangeably.

[0027] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0028] In this document, the term “and / or” includes any and all combinations of one or more of the listed related items.

[0029] In this article, the term "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0030] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0031] In semiconductor manufacturing, hundreds or even thousands of process steps can be categorized into specific process types (also known as process step types) based on physicochemical principles and functional attributes, such as photolithography, etching, thin film deposition, ion implantation, chemical mechanical polishing, and cleaning. Furthermore, each process step strictly adheres to a predetermined process sequence (also known as the step's position on the production line) within the overall manufacturing process. For example, the process sequence can be the number of steps from the start or end of the process flow, or it can be pre-defined according to functional partitions, assigning process steps to different sequential positions, such as a front-end position at the start of the process, a middle-end position in the middle of the process, or a back-end position at the end of the process.

[0032] In this article, "machine" refers to equipment on a production line that performs specific process steps. For example, in a semiconductor wafer fab, it can specifically refer to lithography machines, etching machines, thin film deposition equipment, ion implanters, or chemical mechanical polishing equipment. Machine data is collected and recorded during the operation of each machine for real-time monitoring, fault diagnosis, yield analysis, and predictive maintenance.

[0033] For example, machine data may include engineering data on the health status of the equipment itself (such as time-series data such as temperature, pressure, gas flow, motor speed, vibration spectrum, etc.), process parameters of the processing (such as recipe settings, actual execution values, alarm logs, event records), and contextual information related to production results (such as wafer ID, batch number, process step label, process sequence label, timestamp, etc.), etc., without limitation.

[0034] It should be understood that the sensitivity and importance of data from different types of process steps vary in subsequent yield analysis. Furthermore, the logical order of each step on the production line directly determines the time span between data generation and final test feedback, leading to different application requirements for various types of data. Specifically, for process steps with higher sensitivity and / or importance, even minor fluctuations in their data often have a decisive impact on the final yield, thus requiring frequent backtracking analysis and multi-dimensional correlation assessments throughout the product lifecycle. Similarly, for process steps further upstream in the production process, the time span between data generation and final test results is the longest, resulting in multiple intermediate state verifications, hypothesis testing, and cross-batch comparisons during the long period of waiting for test feedback.

[0035] Based on this, embodiments of this application provide a method, device, and storage medium for managing machine data, specifically proposing a dynamic data recompression mechanism based on timeliness prediction and memory adaptation. By leveraging the technological characteristics of machine data, compression strategies covering different time intervals throughout its entire lifecycle are planned. As the data lifecycle naturally evolves, different compression strategies are automatically invoked and recompression operations are performed, thereby achieving dynamic, on-demand allocation of storage resources and achieving a balance between data storage cost and response efficiency.

[0036] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0037] In some embodiments, to optimize storage resource configuration, machine data can have a pre-defined data lifecycle. This data lifecycle includes multiple lifecycle stages such as data generation, active use, archiving, and eventual destruction or permanent retention. A data lifecycle stage table can be pre-configured to record the complete time span of the data lifecycle stage table for different process steps, as well as the window division for each stage. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of a future timeline provided in an embodiment of this application, such as... Figure 1 As shown, the lifecycle of machine data generated by key etching process steps can be dynamically divided into: the monitoring period from day 0 to day 7; the active period from day 8 to day 90; the waiting period from day 91 to day 180; and the archiving period for periods exceeding 180 days.

[0038] In some embodiments, the window length and complete time span of each stage in the data lifecycle are dynamically set according to the type of process step and its sequential position in the production line. The specific type of lifecycle stage can be flexibly set according to the actual production line situation and application requirements. For example, critical upstream processes, due to their long impact chain, have longer waiting periods and longer complete time spans compared to subsequent processes; while non-critical downstream processes have significantly shortened waiting periods, quickly entering the archiving period to achieve optimal allocation of storage resources. As another example, process steps with higher sensitivity and / or importance have longer complete time spans, allowing machine data to be retained for a longer period.

[0039] In some embodiments, to optimize storage resource allocation, the machine data can be preset with an analysis and evaluation window. This window refers to a specific time interval after the machine data is generated, a period typically used or critical for yield root cause analysis, anomaly pattern identification, and process drift detection. For example, for the etching process step described above, considering the cumulative time consumed by subsequent thin film deposition, photolithography, and testing processes, such as... Figure 1As shown, the optimal analysis and evaluation window is precisely locked between the 7th and 60th day after data generation (or the 1st to 8th week), for post-etching defect / morphology root cause analysis, CD deviation analysis, and yield anomaly tracing.

[0040] In some embodiments, the analysis and evaluation window is the period during which the machine data is used for quality analysis and / or yield assessment. It can be an analysis and evaluation window determined based on historical experience statistics or prior knowledge graphs for different process categories and / or process sequences, and can be flexibly set according to the actual production line situation and application requirements. For example, when the target process is photolithography, located in the front-end of the production line (FEOL / MOL stage), the corresponding analysis and evaluation window is from day 3 to day 30 after data generation, used for overlay / CD / defect rapid feedback, EPE (Edge Placement Error) analysis, and yield improvement. As another example, when the target process is thin film deposition / CMP, located in the middle stage of the production line, the corresponding analysis and evaluation window is from day 10 to day 90 after data generation, used for thickness uniformity / flatness / stress root cause analysis and cross-layer impact assessment. For example, when the target process is the final metal etching / dielectric layer, which is located in the back row of the production line, the corresponding analysis and evaluation window is from day 1 to day 15 (or shorter) after data generation, for rapid quality analysis and traceability of electrical anomalies.

[0041] Furthermore, it is worth noting that, unlike conventional industrial data, wafer fabrication involves numerous processes and requires high precision. Therefore, even minute parameter fluctuations can affect the final yield, resulting in a massive amount of data points to be analyzed. This extremely high precision requirement also necessitates timely monitoring and maintenance of the production line by engineers. Consequently, this demand for real-time analysis with a vast amount of data presents significant challenges to wafer production monitoring and analysis. To address this, this application aims to provide a data management method to reconcile the conflict between the real-time nature of data retrieval and the pressure of data storage.

[0042] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating a machine data management method provided in an embodiment of this application, such as... Figure 2 As shown in the figure, this application provides a method for managing machine data.

[0043] S101, obtain the target process step executed by the target machine, and the machine data generated during the execution of this process step (such as the target process step).

[0044] S102, determine the timeliness information of the machine data according to the process category and / or process sequence of the target process step.

[0045] The timeliness information is used to characterize the machine data generated at a certain moment or time period, and the reading demand at different nodes of the future timeline. In some embodiments, the timeliness information includes at least one of: a lifecycle stage window, an analysis and evaluation window, and a preset call window.

[0046] In some embodiments, standardized timeliness information, including lifecycle stage windows and the start and end times of analysis and evaluation windows, can be predefined according to the process category and / or process sequence of the process steps. When new machine data is generated, the corresponding timeliness information is automatically matched by parsing the process step labels and process sequence labels in the machine data, without the need for manual intervention or real-time calculation.

[0047] In some embodiments, the preset call window refers to the time interval during which machine data is predicted to be accessed frequently based on the data access records of associated machines.

[0048] In some embodiments, a machine profile is constructed, which includes at least one of machine type and process technology. Based on the machine profile, multiple machines are clustered into one or more machine groups. The machine profile may optionally include one or more of the following: basic equipment attributes (such as equipment manufacturer, specific model, core hardware architecture version, etc.), execution process attributes (such as the process node to which it belongs, the specific process steps executed, key process parameter windows, etc.), and operating environment attributes (such as temperature, humidity, etc.).

[0049] In some embodiments, machines are clustered based on their basic equipment attributes, process attributes, and operating environment attributes to obtain different types of machine groups. The machines in a group can be distributed across one or more production lines, and their physical attributes and process functions are highly homogeneous. Furthermore, the inclusion of machines with similar functions but significant differences in internal structure or control logic is avoided, ensuring that the operating performance of each machine within the group has good comparability and reference value.

[0050] In some embodiments, a target machine group is obtained, which includes the target machine and several other machines of the same type as the target machine.

[0051] For example, the other machines include several concurrent machines whose usage time differs from the target machine by no more than a preset time. These include machines in the same production batch or at the same life cycle stage as the target machine, such as those all operating between the 30th and 40th day after production commencement. The preset time can be flexibly set according to the actual application scenario, such as 10 days.

[0052] For example, the other machines include those with a longer service life than the target machine. It should be understood that "previous machines" refers to other machines belonging to the same equipment type and process step as the target machine, but with a longer cumulative operating life (or longer data lifecycle) than the target machine. Because of their longer operating life, the historical databases of these earlier machines fully record data access logs from new machine commissioning to the current aging stage. These machine data access logs can serve as a time-predictive sample of the target machine's future state, and their past data access patterns have certain reference and predictive value.

[0053] In some embodiments, based on the data access logs of the prior machine when it reached the same service life as the target machine, the peak time period when the target machine's data will be accessed in the future is determined, and the peak time period is used as a preset call window.

[0054] Specifically, using the current runtime of the target machine as a baseline, the data access logs of previous machines reaching the same runtime node are retrieved back to determine the peak time period (e.g., days 8-15) within a future preset associated time period (e.g., one month). This peak time period is then mapped onto the target machine's future timeline to obtain a preset access window. This leverages the common behavioral characteristics of similar machine groups at the same lifecycle stage, overcoming the limitation of predicting future access patterns for a single new machine. Before the predicted peak period arrives, the compression ratio of the target machine's data is adjusted to reduce the impact of read latency.

[0055] For example, equipment A, B, and C, all classified as "dry etching equipment" and using the "7nm FinFET" process, are grouped into the same equipment group. Equipment A is the target equipment, which has been running for 10 months. Equipment B is the existing equipment, which has been running for 24 months. In its 12th month, a plasma anomaly caused its data access frequency to reach as high as 1500 times / day. Based on this, it is predicted that the second month of the future timeline will be the peak period for data access to the target equipment, and this is set as the preset access window.

[0056] S103, based on the start and end times of the window in the timeliness information, the future timeline is divided into several time intervals. Preferably, these can be multiple consecutive and non-overlapping time intervals.

[0057] For example, the start and end times of various windows in the timeliness information are extracted, including the switching times of each stage of the data lifecycle, and the start and end times of the analysis and evaluation window and the preset call window, such as... Figure 1 The time points are 0, 7, 60, 90, and 180 days. These time points are sorted chronologically and deduplicated to form a series of ordered division points. The time intervals between adjacent division points constitute a continuous and non-overlapping time interval, such as... Figure 1In the range [0, 7), [7, 60), [60, 90), [90, 180), [180, ∞), the active period is divided into two time intervals due to the analysis and evaluation window, thus ensuring that the access probability is consistent within each time interval.

[0058] Alternatively, in some embodiments, one type of timeliness information can be selected as the time axis division condition.

[0059] It should be understood that the data popularity characteristics at several time points within each time interval are consistent, and a unified storage strategy can be used for management. This partitioning mechanism avoids strategy fragmentation caused by making individual decisions for each time point, and improves the efficiency and stability of subsequent dynamic recompression execution.

[0060] S104. Based on the predicted access probability corresponding to different time-sensitive information, calculate the comprehensive access probability of each time interval, and assign a corresponding compression strategy to each time interval based on the comprehensive access probability.

[0061] Specifically, please refer to Figure 3 , Figure 3 This is a schematic diagram of a comprehensive access probability generation architecture provided in an embodiment of this application, such as... Figure 3 As shown, the lifecycle stage window, analysis and evaluation window, and preset call window in the timeliness information each correspond to different predicted access probabilities (also known as probability components). Based on the predicted access probabilities, the comprehensive access probability of the machine data in different time intervals is determined to characterize the overall expected data popularity in that interval. Furthermore, compression strategies with different compression ratios are invoked according to the comprehensive access probability. The degree of compression of the machine data by the compression strategy is negatively correlated with the comprehensive access probability.

[0062] It should be understood that a higher overall access probability indicates a greater likelihood of the machine data being accessed during that time period. Therefore, a lower compression level is allocated to the data, resulting in the use of more storage resources and minimizing data recovery latency to meet the demands of time-sensitive business operations. Conversely, a lower overall access probability indicates that the machine data is in a low-intensity state during that time period. Therefore, a higher compression level is allocated, resulting in the use of fewer storage resources. During this period, the sparse access demand allows for a greater tolerance for data recovery latency. This achieves a dynamic balance between storage cost and response efficiency, thereby adapting storage resource allocation to business needs.

[0063] In some embodiments, S104 includes: querying a preset data lifecycle stage table to determine a first predicted probability for a time interval. The data lifecycle stage table records window divisions of lifecycle stages and associates them with a base access probability value (i.e., the first predicted probability). For example, the first predicted probability decreases as the time difference increases; for instance, the probability of newly generated data within 7 days (i.e., the monitoring period) is 0.9, decreasing to 0.7 from days 8-90 (i.e., the active period), further decreasing to 0.4 from days 91-180 (i.e., the waiting period), and decreasing to 0.1 after 180 days (i.e., the archiving period).

[0064] In some embodiments, S104 includes: obtaining a second predicted probability within the time interval of the analysis and evaluation window, such as when the analysis and evaluation window is from day 7 to day 60, the predicted probability during this period is 0.85.

[0065] In some embodiments, S104 includes: obtaining a third prediction probability within the time interval of the preset call window, such as when the preset call window is from day 10 to day 40, the prediction probability during this period is 0.75.

[0066] In some embodiments, each interval naturally corresponds to a unique time-sensitive label. For example, for a time interval that is simultaneously in an active period and an analysis and evaluation window, the time-sensitive label can be both the active period and the analysis and evaluation window. Alternatively, for a time interval that is only in a waiting period, the time-sensitive label can be the waiting period. At least one predicted access probability can be quickly retrieved and determined for each time interval based on the time-sensitive label.

[0067] In some embodiments, when at least two probability components of the first predicted probability, the second predicted probability, and the third predicted probability are obtained, the at least two probability components are weighted and fused to obtain the comprehensive access probability.

[0068] Specifically, when multiple probability components exist at any given time point, weights are assigned based on business priorities, and the predicted probability components with different business semantics are weighted and fused to obtain a comprehensive access probability that better meets actual needs. For example, the analysis and evaluation window directly supports root cause localization and yield improvement; the weight of the second predicted probability can be set to the highest; the preset call window reflects the group's data reading needs; and the lifecycle serves as the basic rule, with the weight of the first predicted probability greater than that of the third predicted probability, and the weight of the third predicted probability greater than or equal to that of the first predicted probability.

[0069] For example, on the 20th day, there are three probability components: the first predicted probability has a weight of 0.25, the second predicted probability has a weight of 0.45, and the third predicted probability has a weight of 0.3. In this case, the overall access probability is (0.7×0.25)+(0.85×0.45)+(0.75×0.3)=0.7825.

[0070] In some embodiments, when any one of the first predicted probability, the second predicted probability, and the third predicted probability is obtained, the corresponding probability component is used as the comprehensive access probability.

[0071] S105 applies corresponding compression strategies to compress and store machine data in different time intervals.

[0072] In some embodiments, S105 includes: pre-establishing a mapping relationship between the overall access probability and the compression strategy. The mapping relationship includes: a preset number of levels, a probability threshold range corresponding to each level, and a compression strategy associated with each probability threshold range.

[0073] For example, in some embodiments, the mapping relationship can be used to define the size of the probability interval corresponding to different levels for a certain number of levels, and the compression strategy associated with each probability interval.

[0074] For example, different comprehensive access probability datasets can be divided into different probability threshold ranges (e.g., setting different numbers of levels).

[0075] Specifically, the mapping relationship divides the overall access probability into multiple probability threshold intervals (such as high, medium-high, medium, medium-low, low, etc.), and each probability threshold interval corresponds to a specific compression strategy. In some embodiments, the compression strategy includes multiple compression algorithms (such as lossless compression algorithms LZ4 and Zstandard) and their corresponding compression ratios, which can achieve different degrees of compression for machine data, thereby achieving flexible adaptation between storage cost and response speed.

[0076] When machine data is in a high-probability access period, more storage resources are automatically allocated to hot data, and a low-compression strategy is applied. This ensures that massive logs can be accessed instantly during real-time fault diagnosis or yield review, avoiding parsing delays caused by excessive compression, thereby guaranteeing the timeliness and accuracy of production decisions. Conversely, when machine data is in a low-probability access period, a high-compression strategy is automatically switched to cold data, significantly reducing the consumption of storage resources, lowering overall hardware investment and maintenance costs, and achieving multiple optimizations in resource utilization and response speed.

[0077] It should be understood that the finer the tiering, the more precise the compression strategy, allowing for a more accurate match with the actual needs of data at different stages of popularity, thereby optimizing storage resource utilization efficiency. However, more tiers also mean more strategy switching points may occur on the future timeline, leading to an increase in the number of re-compression operations and resulting in additional computing power overhead. Therefore, in actual deployment, the number of tiers needs to be reasonably set based on process sensitivity, storage architecture capabilities, and operational costs. The number of tiers can be greater than or equal to 3, such as 3, 4, 5, or 6.

[0078] For example, in this embodiment, the number of levels is used to define the number of levels in the probability interval.

[0079] In some embodiments, the mapping relationship includes: when the overall access probability of a time interval is higher than a first probability threshold, a first compression strategy is adopted; when the overall access probability of a time interval is lower than or equal to the first probability threshold and higher than a second probability threshold, a second compression strategy is adopted; when the overall access probability of a time interval is lower than or equal to the second probability threshold, a third compression strategy is adopted; wherein, the compression degree of the first compression strategy is lower than that of the second compression strategy, and the compression degree of the second compression strategy is lower than that of the third compression strategy.

[0080] The first probability threshold and the second probability threshold can be flexibly set according to actual application requirements. For example, the first probability threshold can be set to 0.85-0.95, preferably 0.9; the second probability threshold can be set to 0.65-0.75, preferably 0.7.

[0081] For example, the first compression strategy uses the LZ4 algorithm with a compression ratio of approximately 2–4:1 and a decompression speed of 3000–6000 MB / s; the second compression strategy uses the Zstandard algorithm with a compression ratio of approximately 4–8:1 and a decompression speed of 1000–2000 MB / s; and the third compression strategy uses the Gorilla algorithm with a compression ratio of approximately 8–15:1 and a decompression speed of 500–1500 MB / s.

[0082] In some embodiments, compression strategies for adjacent time intervals are compared, and when the compression strategies are the same, adjacent time intervals can be merged into one time interval.

[0083] Specifically, when time progresses to the beginning of a certain time interval, the application of the corresponding compression strategy is automatically triggered. The compression operations of each time interval are independent of each other and sequentially connected, and the strategy switch only occurs at the interval boundary to avoid frequent re-compression.

[0084] It should be understood that the embodiments of this application perform differentiated compression and storage operations on the machine data within the corresponding time period, so that the machine data is always saved with a compression intensity that matches its access frequency at different stages of its life cycle, thereby maximizing long-term storage efficiency while ensuring read performance during critical periods.

[0085] In some embodiments, the method further includes: acquiring in real time the measured access frequencies of multiple machines in the target machine group within a preset time period; calculating the group access frequency based on the measured access frequencies of the multiple machines; and updating the compression strategy for the current time interval based on the group access frequency when abnormal fluctuations in the group access frequency are detected.

[0086] The preset time period is a time window that traces back from the current point in time and can be flexibly set according to different process types. If the target process is photolithography, the impact of photolithography events is more immediate, so the preset time period can be 2 days; if the target process is post-processing metallization / CMP, the electrical feedback cycle is longer, so the preset time period can be extended to 5 days.

[0087] Specifically, during operation, the system collects real-time measured access frequencies of all members in the target machine group (including earlier machines and concurrent machines) within a preset time period, aggregates and statistically analyzes these frequencies, and generates the current group access frequency. For example, a weighted average method is used, assigning higher weights to access records closer to the current time to highlight the timeliness of recent behavior; another example is dynamically allocating weights based on the similarity between the machine and the target machine. For instance, concurrent machines, being in the same lifecycle stage, have more valuable real-time access behavior for the target machine, thus receiving a higher weight than earlier machines.

[0088] When abnormal fluctuations in the frequency of group access are detected (such as the magnitude or standard deviation of the change exceeding the preset fluctuation threshold), it is determined that there may be a common analysis need or potential process abnormalities (such as multiple lithography machines of the same type being frequently accessed for yield review in a short period of time), and the compression strategy is updated to an instant compression strategy.

[0089] It should be understood that in the early stages of a systemic deviation in a certain type of equipment, some sensitive machines or machines that have experienced the same operating condition first will be the first to trigger data analysis needs. As a member of the group, the target machine is likely to face the same analysis scenario soon. By reducing the compression level of the relevant data of the target machine in advance, it is ensured that the key data of the target machine is ready before potential common process problems occur, which shortens the waiting delay during fault diagnosis and improves the production line's response speed to systemic risks.

[0090] In some embodiments, the method further includes updating the compression strategy for the current time interval based on the measured access frequency of the target machine.

[0091] Specifically, during operation, the system collects the measured access frequency of the target machine within a preset time period in real time, aggregates and statistically analyzes this data, and generates the current local access frequency. For example, a weighted average method is used, assigning higher weights to access records closer to the current time to highlight the timeliness of recent behavior. When abnormal fluctuations in the local access frequency are detected (such as changes exceeding a preset fluctuation threshold), it is determined that there may be a sudden process anomaly requiring high-frequency access to machine data, and the compression strategy is updated to an immediate compression strategy.

[0092] In some embodiments, the compression strategy and compression ratio of the real-time compression strategy can be preset, generally to light compression, such as the first compression strategy. That is, even if the current time interval was originally planned to use a high compression strategy, the dynamic update mechanism of the compression strategy is triggered: the compression level is temporarily downgraded to light compression.

[0093] In some embodiments, the adjustment of the compression strategy only applies to the current and adjacent short-term time interval (e.g., 10 days). Once the group access frequency or the measured access frequency of the target machine returns to normal levels, it automatically reverts to the original compression strategy. For example, the specific length of the short-term time interval and the normal level frequency standard can be set by the user (e.g., an engineer).

[0094] In some embodiments, the compression strategy for machine data throughout its entire lifecycle is pre-planned and predictable. Based on the current storage status and future strategy evolution, the trend of cumulative memory usage is calculated, thereby identifying potential risks of sudden dynamic changes in storage space in advance, and thus establishing an adaptive update mechanism for mapping relationships.

[0095] For example, the method further includes: predicting the cumulative memory usage at different time points in the future time axis based on the compression strategy of multiple batches of machine data within the target machine; when the cumulative memory usage at any time point is greater than a preset memory threshold (identifying the time point as an abnormal time point), updating the mapping relationship between the comprehensive access probability and the compression strategy at the corresponding abnormal time point, including: reducing the number of levels in the mapping relationship; and updating the compression strategy of the time interval where the abnormal time point is located based on the updated mapping relationship.

[0096] The cumulative memory usage refers to the total storage space expected to be occupied by all batches of machine data generated by the target machine and still within its retention period at a future point in time, under the current compression strategy. This value changes dynamically as new data is continuously generated and old data is gradually compressed or archived. Correspondingly, the preset memory threshold refers to the preset upper limit of storage capacity, used to prevent storage resources from being overloaded due to excessively rapid data accumulation. The specific value can be flexibly set according to actual needs.

[0097] Specifically, based on the compression strategies already allocated to data from each batch of machines, the cumulative memory usage at each point in time is projected along the future timeline. If the predicted usage at a certain point in time will exceed a preset memory threshold, a risk of storage overload is identified. At this time, the mapping relationship between the overall access probability and the compression strategy corresponding to that abnormal time point is automatically optimized: for example, reducing the number of levels (e.g., merging from 5 levels to 3 levels), so that more low-to-medium-frequency data is categorized into strategies with higher compression ratios. Subsequently, based on the updated mapping relationship, the compression strategy for the time interval containing that time point is reallocated, forcibly increasing the overall compression intensity. While ensuring the availability of critical high-frequency data, this proactively prevents storage resource exhaustion, achieving coordinated management of performance and capacity.

[0098] In some embodiments, the updated mapping relationship includes: when the overall access probability of a time interval is higher than a first probability threshold, a first compression strategy is adopted; when the overall access probability of a time interval is lower than or equal to the first probability threshold, a third compression strategy is adopted.

[0099] In some embodiments, the future timeline is sampled at a preset time granularity, which is one of n hours, n days, n weeks, or n months, where n is a positive number. The preset time granularity defines the interval unit for sampling and deciding on access probabilities, thereby improving the evaluation accuracy and execution rhythm of the recompression mechanism. This avoids computational redundancy and storage overhead caused by indiscriminate intensive sampling over continuous time, effectively improving the overall system operating efficiency. It should be understood that using a coarser time granularity (such as weekly or monthly) can significantly reduce the number of recompression triggers. Only at the end of each granularity period (such as the end of each week or month) is the data access probability uniformly evaluated and a decision made on whether to perform recompression made, avoiding high-frequency, small-batch compression operations, thereby reducing the computational resource consumption of the storage system. Conversely, finer granularity (such as daily) can more sensitively capture the phased changes in data popularity throughout its lifecycle, thus triggering recompression at the appropriate time.

[0100] The method described in this application can be used in a wide variety of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0101] Please see Figure 4 , Figure 4 This is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application. The computer device may be a terminal device or a server.

[0102] For example, the above method can be implemented as a computer program, which can be used in, for example... Figure 4 It runs on the computer device shown.

[0103] like Figure 4 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0104] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any machine data management method and its specific implementation steps.

[0105] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0106] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any machine data management method and the specific implementation steps of the method.

[0107] This network interface is used for network communication, such as sending assigned tasks.

[0108] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0109] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: S101, Obtain the target process step executed by the target machine, and the machine data generated during the execution of this process step; S102, determine the timeliness information of the machine data according to the process category and / or process sequence of the target process step, wherein the timeliness information includes at least one of the following: life cycle stage window, analysis and evaluation window, and preset call window; S103, based on the start and end times of the window in the timeliness information, the future timeline is divided into several continuous and non-overlapping time intervals; S104, based on the predicted access probabilities corresponding to different time-sensitivity information, calculate the comprehensive access probability for each time interval, and assign a corresponding compression strategy to each time interval based on the comprehensive access probability; wherein, the compression degree of the compression strategy on the machine data is negatively correlated with the comprehensive access probability. S105 applies corresponding compression strategies to compress and store machine data in different time intervals.

[0110] For example, the processor is used to run a computer program stored in the memory, and is also used to implement the steps and specific implementation steps of the machine data management method provided in any embodiment of this application, which will not be repeated here.

[0111] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement the steps and specific implementation steps of the machine data management method provided in any of the embodiments of this application.

[0112] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0113] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for managing machine data, characterized in that, The method includes: S101, Obtain the target process steps executed by the target machine, and the machine data generated during the execution of the target process steps; S102, determine the timeliness information of the machine data according to the process category and / or process sequence of the target process step, wherein the timeliness information includes at least one of the following: life cycle stage window, analysis and evaluation window, and preset call window; S103, based on the start and end times of the window in the timeliness information, the future timeline is divided into several continuous and non-overlapping time intervals; S104, based on the predicted access probabilities corresponding to different time-sensitivity information, calculate the comprehensive access probability of each time interval, and assign a corresponding compression strategy to each time interval based on the comprehensive access probability; wherein, the compression degree of the compression strategy on the machine data is negatively correlated with the comprehensive access probability. S105, apply corresponding compression strategies to compress and store machine data in different time intervals; Before performing compressed storage, the compression strategy is updated, which includes: Based on the compression strategy of multiple batches of machine data within the target machine, predict the cumulative memory usage at different time points in the future time axis; When the cumulative memory usage at any point in time exceeds a preset memory threshold, update the mapping relationship between the comprehensive access probability and the compression strategy at the corresponding abnormal time point, including: reducing the number of levels in the mapping relationship; Based on the updated mapping relationship, the compression strategy for the time interval where the abnormal time point is located is updated.

2. The method according to claim 1, characterized in that, S104 includes: Query the preset data lifecycle stage table to determine the first prediction probability of the time interval; and / or, obtain the second prediction probability of the time interval located in the analysis and evaluation window; and / or, obtain the third prediction probability of the time interval located in the preset call window. When at least two probability components of the first prediction probability, the second prediction probability, and the third prediction probability are obtained, the at least two probability components are weighted and fused to obtain the comprehensive access probability. When any one of the first, second, and third prediction probabilities is obtained, the corresponding probability component is used as the comprehensive access probability.

3. The method according to claim 1, characterized in that, S104 includes: Establish a mapping relationship between overall access probability and compression strategy in advance; The mapping relationship includes: a preset number of levels, a probability threshold range corresponding to each level, and a compression strategy associated with each probability threshold range.

4. The method according to claim 3, characterized in that, S104 includes: When the overall access probability of a time interval is higher than the first probability threshold, the first compression strategy is adopted; When the overall access probability of a time interval is lower than or equal to the first probability threshold and higher than the second probability threshold, the second compression strategy is adopted. When the overall access probability of a time interval is lower than or equal to the second probability threshold, the third compression strategy is adopted. The compression level of the first compression strategy is lower than that of the second compression strategy, and the compression level of the second compression strategy is lower than that of the third compression strategy.

5. The method according to claim 1, characterized in that, The method further includes: Obtain a target machine group, which includes the target machine and several other machines of the same type as the target machine, including prior machines with a service life longer than the target machine; Based on the data access logs of the existing machine when it reached the same service life as the target machine, the peak time period when the target machine's data will be accessed in the future is determined, and the peak time period is used as the preset call window.

6. The method according to claim 5, characterized in that, The method further includes: Real-time acquisition of the measured access frequency of multiple machines in the target machine group within a preset time period; The group access frequency is calculated based on the measured access frequencies of multiple machines; when abnormal fluctuations in the group access frequency are detected, the compression strategy for the current time interval is updated based on the group access frequency.

7. The method according to claim 5, characterized in that, The compression strategy for the current time interval is updated based on the measured access frequency of the target machine.

8. The method according to claim 6 or 7, characterized in that, The updated mapping relationships include: When the overall access probability of a time interval is higher than the first probability threshold, the first compression strategy is adopted; When the overall access probability of a time interval is lower than or equal to the first probability threshold, a third compression strategy is adopted.

9. A computer device, characterized in that, The device includes: Memory, used to store computer programs; A processor is configured to execute the computer program and, in executing the computer program, implement the machine data management method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the machine data management method as described in any one of claims 1 to 8.

Images