Chip design and manufacturing optimization system and method based on edge reinforcement learning

Abstract

The present invention discloses a chip design and manufacturing optimization system and method based on edge reinforcement learning, which involves the field of data analysis and includes a scene recognition module, a strategy selection module, a decision correction module, and a cloud update module. The present invention realizes robust recognition of the equipment operating status through a state-aware feature channel and a confidence scoring mechanism, and introduces a fuzzy recognition and alarm mechanism to enhance adaptability; with the help of a dual-index strategy library and an impact gradient transmission mechanism, it realizes inter-process linkage optimization and improves process synergy; through the dynamic upload of state quadruple and cloud-based oscillation identification, the system can automatically enter a cool-down period lock under policy anomalies and energy consumption fluctuations, effectively suppressing the spread of anomalies. At the same time, lightweight edge deployment and reinforcement learning mechanisms enable it to have high response, low communication and self-learning capabilities, suitable for large-scale complex chip manufacturing environments.

Description

Technical Field

[0001] The present invention relates to the field of data analysis, and specifically to a chip design and manufacturing optimization system and method based on edge reinforcement learning. Background Art

[0002] Semiconductor chip manufacturing is a highly complex process involving hundreds of precise steps, primarily divided into stages such as wafer preparation, pattern transfer, doping, thin film deposition, etching, and packaging and testing. Modern chip manufacturing can take months to complete. These processes require extremely specialized equipment, including photolithography machines, etching systems, chemical vapor deposition (CVD) equipment, and various metrology and measurement instruments. Traditional industrial control systems often use fixed rules or models, lacking adaptive learning capabilities and difficulty automatically adjusting decision-making strategies based on environmental changes. Existing reinforcement learning applications primarily focus on a single process or piece of equipment and lack the ability for collaborative learning among multiple agents.

[0003] Prior art, such as the invention patent with publication number CN118194790B, relates to a chip design method and a chip design system. The chip design method includes the following steps: reading register transfer level code data; identifying multiple register transfer level codes in the register transfer level code data to classify multiple first registers and multiple second registers corresponding to the multiple register transfer level codes, wherein the multiple first registers are not electrically connected to an interface and the multiple second registers are electrically connected to multiple interfaces; performing multi-bit register merging on the multiple first registers to generate at least one first multi-bit register; and performing multi-bit register merging on the multiple second registers based on initial physical location information of the multiple second registers to generate at least one second multi-bit register.

[0004] The prior art, such as the invention patent with announcement number CN114818553B, is a chip integration design method, which includes: defining the hierarchical structure of each chip module in a template; setting the RTL file path of the module in the template, and setting the module name, and whether it belongs to the module for which RTL code is to be generated; analyzing the RTL file of the module for which RTL code has been generated, and extracting the port connection information and parameter information of the module; receiving the connection information of the unconnected ports between modules added by the user in the template and the instantiation parameter values of the module; using a script tool to analyze the added template, and generating the RTL code of the module for which RTL code is to be generated; generating corresponding port connections for the module for which port connection information has been defined, and instantiating the module for which parameter values have been defined using the defined parameter values.

[0005] Based on the above solutions, it can be seen that existing solutions are highly dependent on manual maintenance and have the risk of design errors or performance overload. In addition, traditional centralized control systems need to transmit all data to a central server for processing, which cannot meet the needs of real-time control in industrial sites and mostly use fixed rules or models. Therefore, a chip design system that can be flexibly controlled and responsive is needed. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention provides a chip design and manufacturing optimization system and method based on edge reinforcement learning. To achieve the above objectives, the present invention is implemented through the following technical solutions: The chip design and manufacturing optimization system based on edge reinforcement learning includes:

[0007] The scene recognition module is used to collect raw data on equipment operation during chip manufacturing processes, and to perform state feature extraction and scene classification to obtain scene labels for the process equipment.

[0008] The strategy selection module is used to select the corresponding strategy model from the local strategy library as the adjustment strategy model according to the scene label through the edge agent of the process equipment after the scene recognition is completed.

[0009] The decision correction module is used to calculate the impact gradient of the adjustment strategy model on the downstream process equipment through the edge agent of the process equipment. At the same time, the edge agent of the downstream process equipment introduces the impact gradient as a penalty factor in the decision-making process to make decision corrections.

[0010] The cloud update module is used to continuously update the local state quadruple to the cloud through the edge agent of the process equipment. The cloud identifies whether there is policy oscillation and energy consumption anomaly. If so, the process equipment is controlled to enter the cooling-off period lock mode.

[0011] Chip design and manufacturing optimization methods based on edge reinforcement learning include:

[0012] The original data of equipment operation is collected on the process equipment of chip manufacturing, and the state feature extraction and scene classification are performed to obtain the scene labels of the process equipment.

[0013] After the scene recognition is completed, the edge agent of the process equipment selects the corresponding policy model from the local policy library as the adjustment policy model according to the scene label.

[0014] The edge agent of the process equipment calculates the impact gradient of the adjustment strategy model on the downstream process equipment. At the same time, the edge agent of the downstream process equipment introduces the impact gradient as a penalty factor in the decision-making process to make decision corrections.

[0015] The edge agent of the process equipment continuously updates the local state quadruple to the cloud. The cloud identifies whether there are policy fluctuations and energy consumption anomalies. If so, the process equipment is controlled to enter the cooling-off period lock mode.

[0016] Compared with the prior art, the embodiments of the present invention have at least the following beneficial effects:

[0017] (1) The present invention provides a chip design and manufacturing optimization system based on edge reinforcement learning. By deploying state-aware feature channels on the process equipment side, it can achieve accurate collection and structured processing of raw data from equipment operation. By combining with the equipment model feature channel mapping library, it can extract highly discriminative state features and use a confidence scoring mechanism to quantitatively identify multiple candidate scenarios, effectively avoiding misidentification and weak label interference. When the confidence score does not meet the standard or there are multiple optimal labels, a fuzzy recognition and alarm mechanism is introduced to provide more robustness and fault tolerance for downstream decision-making, ensuring the system's adaptability and accuracy to complex and changing working conditions.

[0018] (2) The present invention uses edge agents to establish dual-index anchors in the local policy library based on the identified scene labels, achieving accurate loading of low-latency policy models; under fuzzy recognition, a multi-model fusion mechanism is introduced to improve generalization. At the same time, the system designs an "influence gradient" evaluation and compression transmission mechanism for downstream process equipment, so that cross-node decision corrections can be achieved between process equipment based on feedback factors, forming an edge linkage mode that takes into account local independence and system synergy. This mechanism effectively improves the process coordination between equipment and reduces the probability of local optimality and policy conflict problems.

[0019] (3) The present invention achieves dynamic tracking of the long-term operating status of the device and policy fluctuations through continuous updating of the state quadruple and cloud-based policy oscillation identification. When abnormal changes in the policy network or abnormal energy consumption are detected, an alarm and remote intervention are issued, the length of the cooling-off period is automatically calculated, and the edge adjustment policy is frozen to prevent the problem from spreading. An emergency upload mechanism for channel optimization is designed to ensure timely and reliable reporting of abnormal status. This gives the system excellent self-stabilization capabilities, energy consumption management capabilities, and operation and maintenance controllability, making it particularly suitable for large-scale, multi-node, and complex process chain chip manufacturing scenarios.

[0020] (4) By deploying lightweight intelligent agents, the present invention enables decision-making to be completed directly near the data source, significantly reducing communication latency. This enables the system to meet the millisecond-level response requirements of chip manufacturing. By continuously learning from experience through reinforcement learning and continuously optimizing decision-making strategies, the present invention can cope with the emergence of various defects and interferences, and improve the system's adaptability to process changes.

[0021] Of course, any product implementing the present invention does not necessarily need to achieve all of the above advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the system module of the present invention.

[0023] Figure 2 Schematic diagram of the method of the present invention.

[0024] Figure 3 It is a logical flow diagram of the present invention.

[0025] Figure 4 This is a page diagram of a process list of a chip manufacturing operation and maintenance system according to an embodiment of the present invention.

[0026] Figure 5 This is a real-time monitoring page diagram of the chip manufacturing operation and maintenance system involved in an embodiment of the present invention.

[0027] Figure 6 This is a scene identification page diagram of the chip manufacturing operation and maintenance system involved in an embodiment of the present invention.

[0028] Figure 7 This is a diagram of the chip manufacturing operation and maintenance system policy management page involved in an embodiment of the present invention.

[0029] Figure 8 This is a diagram of the cool-down period control page of the chip manufacturing operation and maintenance system involved in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] In the description of the present invention, it should be understood that the terms "opening", "upper", "lower", "thickness", "top", "middle", "length", "inside", "around" and the like indicating orientation or positional relationship are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0032] See also Figure 1 As shown, the embodiment of the present invention provides a chip design and manufacturing optimization system based on edge reinforcement learning, and also refers to Figure 3The figure shows a logical flow diagram of an embodiment of the present invention. The overall process starts with device data collection and ultimately determines whether to enter a cooling-off period based on whether significant anomalies exist, thereby ensuring the stability of device operation and the reliability of the policy model. It includes:

[0033] In an embodiment of the present invention, a display page of a chip manufacturing operation and maintenance system is provided, such as Figure 4 The figure shows a process list page of a chip manufacturing operation and maintenance system according to an embodiment of the present invention, which is used for managers to have an overview of the entire chip manufacturing process and provide corresponding interaction for operation.

[0034] like Figure 5 The figure shows a real-time monitoring page diagram of the chip manufacturing operation and maintenance system involved in an embodiment of the present invention, which is used to monitor relevant information of the current manufacturing process, including original data of equipment operation and scenario confidence scores, etc., to facilitate management personnel to monitor the current manufacturing process in real time.

[0035] like Figure 6 Shown is the scene recognition page of the chip manufacturing operation and maintenance system involved in an embodiment of the present invention, which is used to view the scene recognition process of the edge intelligent agent, including information such as feature extraction of data.

[0036] The scene recognition module is used to collect raw data on equipment operation during chip manufacturing processes, and to perform state feature extraction and scene classification to obtain scene labels for the process equipment.

[0037] Collect raw data from chip manufacturing process equipment, extract state features, and classify scenes to obtain scene labels for the process equipment. The specific process is as follows:

[0038] The original data of equipment operation includes process parameter data, electrical signal data and environmental parameter data.

[0039] It should be noted that in the embodiment of the present invention, since the system is applied to the full-link process in the field of semiconductor chip manufacturing, including wafer preparation, lithography, etching, thin film deposition, doping, metallization and packaging testing, it includes a variety of different types of process equipment. Therefore, when it comes to the acquisition and processing of equipment-related parameters, the embodiment of the present invention will take the lithography equipment parameters as an example to illustrate.

[0040] The original data of equipment operation includes process parameter data, electrical signal data and environmental parameter data. Among them, the process parameter data is read by the control system of the equipment in the embodiment of the present invention, including photolithography light intensity, photolithography time and light source wavelength; the electrical signal data is monitored by the built-in electrical monitoring module, including power supply voltage, power supply current and average power; the environmental parameter data is measured by the environmental monitoring sensor, including the internal temperature of the equipment, average vibration frequency and average vibration intensity.

[0041] Extract the model of the process equipment and match it with the mapping set of equipment model-state perception feature channel pre-stored in the database to obtain the state perception feature channel of the process equipment. Input the original data of the equipment operation into the state perception feature channel of the process equipment to extract the state feature to obtain the preparatory scene labels of the process equipment, including:

[0042] It should be noted that the state perception feature channel involved in the embodiment of the present invention is a set of algorithms and processing flows predefined for different device models, which are used to extract key state indicators from the original data of device operation.

[0043] The state feature extraction process involves preprocessing the raw data from the device's operation and then extracting indicators defined in the state perception feature channel. These indicators include process parameter indicators, electrical signal indicators, and environmental parameter indicators. In this embodiment of the present invention, process parameter indicators include the photolithography light intensity stability coefficient, photolithography time deviation, and light source wavelength fluctuation rate; electrical signal indicators include the current instantaneous mutation rate, average power stability, and voltage overshoot / undershoot rate; and environmental parameter indicators include the internal temperature stability coefficient, average vibration frequency principal component, and average vibration intensity anomaly index. These indicators are used to identify the device's operating status and are stored in a database.

[0044] The statistical, time-domain, and frequency-domain characteristics of the raw data from the computing device's operation are calculated. Statistical characteristics include mean, variance, maximum, and minimum values; time-domain characteristics include instantaneous rate of change, slope of change, and peak-to-valley difference; and frequency-domain characteristics include primary frequency components and power spectral density. These characteristics are combined with the indicators defined in the state-sensing feature channel to derive data characteristics for process parameters, electrical parameters, and environmental parameters. For example, if the mean and peak-to-valley values of the photolithography intensity meet the mean and peak-to-valley values defined in the photolithography intensity stability coefficient, then the process parameters exhibit data characteristics indicating stable photolithography intensity.

[0045] Based on the automatic encoder, the features of process parameters, electrical signals and environmental parameters are integrated to form a comprehensive state feature vector. The multi-label neural network classifier is used to input the comprehensive state feature vector and output the labels of various preparatory scenarios of the process equipment. In the embodiment of the present invention, taking the photolithography equipment as an example, the preparatory scenario labels include but are not limited to stable operation scenarios, unstable light intensity scenarios, voltage overshoot interference scenarios and vibration interference scenarios.

[0046] Extract equipment operation reference data for each preparatory scenario label, including process parameter reference data, electrical signal reference data, and environmental parameter reference data, compare them with the original equipment operation data to obtain a set of equipment operation data deviation values, and perform weighted coupling to obtain a confidence score value for each preparatory scenario label. In this embodiment of the present invention, the original data of the lithography equipment operation is used as an example for calculation, specifically including:

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] in, For the The confidence score of the process parameters of the prepared scene labels, For the The confidence score of the electrical signal parameters of the prepared scene labels, For the The confidence score of the environmental parameters of the prepared scene labels, For the confidence scores of the preparatory scene labels, is the lithography light intensity, is the photolithography time, is the wavelength of the light source, is the supply voltage, is the supply current, is the average power, is the internal temperature of the device, is the average vibration frequency, is the average vibration intensity, For the The photolithography intensity of the prepared scene label, For the The lithography time of the prepared scene labels, For the The wavelength of the light source of the prepared scene label, For the The supply voltage of the prepared scene tag, For the The supply current of a prepared scene tag, For the The average power of the prepared scene tags, For the The internal temperature of the device with the prepared scene tag, For the The average vibration frequency of the preparatory scene labels, For the The average vibration intensity of the preparatory scene labels, is the lithography intensity weighting factor, is the lithography time weighting factor, is the light source wavelength weighting factor, is the supply voltage weighting factor, is the supply current weighting factor, is the average power weighting factor, is the weighting factor for the internal temperature of the device, is the average vibration frequency weighting factor, is the average vibration intensity weighting factor, is the weighting factor of the confidence score value of the process parameter, is the weighting factor of the confidence score value of the electrical signal parameter, is the weighting factor of the environmental parameter confidence score value, Number the labels for the preparatory scenes, , The number of labels for the prepared scenes.

[0052] It should be noted that, in the embodiment of the present invention, a dynamic weight adjuster is configured to dynamically adjust the weights of each target according to the current chip manufacturing scenario and management requirements.

[0053] The lithography intensity weighting factor is used to adjust the contribution of the lithography intensity parameter to the comprehensive features, reflecting its importance to device status identification. The lithography time weighting factor represents the weight of the lithography time parameter in the overall feature fusion, reflecting its relative impact on process quality. The light source wavelength weighting factor is used to measure the importance of the light source wavelength parameter in status analysis. The supply voltage weighting factor reflects the weight of the supply voltage parameter on the device's operational stability. The supply current weighting factor measures the contribution of the current parameter to the electrical status features. The average power weighting factor represents the weight of the average power parameter in the electrical signal category. The device internal temperature weighting factor is used to adjust the influence of the temperature parameter on environmental status identification. The average vibration frequency weighting factor reflects the sensitivity of the vibration frequency to the device's mechanical status. The average vibration intensity weighting factor is used to measure the importance of the vibration intensity parameter in environmental monitoring. The weighting factor of the process parameter confidence score value represents the weighting of the credibility of the entire process parameter category data, reflecting its data quality and stability; the weighting factor of the electrical signal parameter confidence score value is used to adjust the influence of the overall confidence of the electrical signal parameters; the weighting factor of the environmental parameter confidence score value reflects the regulatory effect of the confidence level of the environmental parameter category data on the comprehensive feature fusion.

[0054] It should also be noted that in the embodiments of the present invention, using photolithography equipment as an example, there is a close correlation between process parameter data, electrical signal data, and environmental parameter data. Specifically, process parameter data such as photolithography intensity, photolithography time, and light source wavelength directly determine the quality and precision of the photolithography process, and the stability of these parameters is key to achieving high-precision manufacturing. The supply voltage, supply current, and average power in the electrical signal data reflect the operating status of the equipment's power supply system. Fluctuations in these electrical parameters can directly affect the process parameter performance of the photolithography equipment. For example, abnormalities in voltage or current can lead to instabilities in the photolithography intensity and light source wavelength. Environmental parameter data, including internal equipment temperature, average vibration frequency, and average vibration intensity, reflects the physical environment and mechanical state of the equipment. Environmental changes can affect the electrical system and process through thermal effects or mechanical vibration. For example, temperature increases can cause light source wavelength drift, while abnormal vibrations can cause photolithography intensity fluctuations. Therefore, these three types of parameters influence and constrain each other.

[0055] The confidence score value of each preparatory scene label is compared with the confidence score threshold pre-stored in the database, and the preparatory scene labels with confidence scores greater than or equal to the confidence score threshold are counted as confident scene labels, and the confidence scene label with the largest confidence score value is selected as the scene label of the process equipment.

[0056] If there are two or more confidence scene labels with the largest confidence score values at the same time, it is determined that the fuzzy recognition processing mode needs to be executed, and these confidence scene labels are recorded as fuzzy scene labels.

[0057] If the confidence score values of all preparatory scene labels are less than the confidence score threshold, an alarm will be issued.

[0058] like Figure 7 The figure shows a chip manufacturing operation and maintenance system policy management page diagram involved in an embodiment of the present invention, which is used by administrators to view and manage policies, including the current policy of the device and policy change records.

[0059] The strategy selection module is used to select the corresponding strategy model from the local strategy library as the adjustment strategy model according to the scene label through the edge agent of the process equipment after the scene recognition is completed.

[0060] Select the corresponding policy model from the local policy library as the adjustment policy model. The specific process is as follows:

[0061] After the edge agent on the process equipment receives the scene label, it establishes a secondary index anchor of the process equipment model-scene label, and puts it into the local policy library for indexing. It obtains the policy model corresponding to the secondary index anchor and loads it as the adjustment policy model of the process equipment. The policy model includes policy network parameters and control adjustment parameters.

[0062] In an embodiment of the present invention, the secondary index anchor of the device model-scenario tag is an index mechanism for quickly retrieving and loading local policy models. It is defined by combining the model identifier of the process equipment with the identified scenario tag in a standardized manner to form a unique index identifier. For example, if the model of the lithography equipment is Litho-M3400-Canon, and its current scenario tag is abnormal light source power fluctuation, and the scenario tag is S002, then the secondary index anchor is Litho-M3400-Canon#S002. This anchor is used to quickly locate the corresponding adjustment policy model in the local policy library and implement dynamic loading.

[0063] In an embodiment of the present invention, taking the lithography equipment as an example, the system loads the corresponding adjustment strategy model from the strategy library through the index anchor Litho-M3400-Canon#S002. The adjustment strategy model includes a set of strategy network parameters and a set of control adjustment parameters. The strategy network consists of a three-layer feedforward neural network, with the input being the comprehensive state feature vector and the output being the recommended ratio of adjustable parameters, including the exposure time adjustment ratio, the light source power correction ratio, and the light intensity stabilization filter coefficient. The network performs inference processing through the weight matrix and activation function, and finally forms a decision output through the Softmax function. At the same time, the control adjustment parameters provide directly executable operation instructions, such as recommending increasing the exposure time by 0.12 seconds, increasing the light source power by 8%, and setting the light intensity change filter coefficient to 0.92. Through the above mechanism, the edge agent can quickly and accurately adapt and adjust the state of the lithography equipment.

[0064] If it is determined that the fuzzy recognition processing mode needs to be executed, the fuzzy recognition processing mode will be executed on the process equipment. The specific process includes: establishing each fuzzy index anchor point of the process equipment model-fuzzy scene label, and indexing it in the local policy library, obtaining each policy model corresponding to the fuzzy index anchor point, loading each policy model and averaging each policy model to obtain the adjustment policy model of the process equipment.

[0065] In an embodiment of the present invention, when the process equipment is in fuzzy recognition processing mode, the fuzzy index anchor constructed by the equipment model and fuzzy scene label is a multi-label coupled index structure, which is defined by combining the equipment model and multiple fuzzy scene labels with confidence in a certain order and uniformly encoding them to form a unique index identifier. For example, for a lithography equipment model Litho-M3400-Canon, the current fuzzy scene labels include abnormal light source power fluctuation, early stage light source aging, and frequent voltage fluctuations. The early stage light source aging scene label is identified as S004, and the frequent voltage fluctuation scene label is identified as S007. Then, Litho-M3400-Canon#S002+S004+S007 is constructed as a fuzzy index anchor to retrieve multiple policy models matching this fuzzy state in the local policy library.

[0066] After completing the loading of multiple policy models, model averaging processing will be performed to form a unified adjustment policy model. This processing process includes two levels: at the policy network layer, the weight parameters of the neural network of each policy model are weighted averaged. For example, if the confidence levels corresponding to the three labels are set to 0.5, 0.3, and 0.2 respectively, the weight matrix and bias items at the same structural position in the policy network are weightedly fused to form a new set of network parameters; at the control adjustment parameter layer, the control parameters in different policy models, such as exposure time adjustment, light source correction ratio, temperature control adjustment value, etc., are also weighted and averaged. The unified adjustment policy model finally constructed has the ability to coordinate responses in a fuzzy state, can effectively take into account the adjustment needs of various potential fault scenarios, and improve the adaptive control accuracy and robustness of the system.

[0067] The decision correction module is used to calculate the impact gradient of the adjustment strategy model on the downstream process equipment through the edge agent of the process equipment. At the same time, the edge agent of the downstream process equipment introduces the impact gradient as a penalty factor in the decision-making process to make decision corrections.

[0068] The edge agent of the process equipment calculates the impact gradient of the adjustment strategy model on the downstream process equipment, including:

[0069] The edge agent of the process equipment calls the local proxy model and inputs the control adjustment parameters of the adjustment strategy model. The proxy model outputs the predicted impact value of the control adjustment parameter on the production quality parameters of the downstream process equipment after application. The predicted impact value is differentiated to obtain the impact gradient on the downstream process equipment.

[0070] It should be noted that the edge agent calls a locally deployed lightweight proxy model. This proxy model is a deep network model trained using historical production data and can simulate the parameter linkage relationship between the device and its downstream devices. The input of the proxy model is the control adjustment parameters in the adjustment strategy model.

[0071] During the specific calculation process, the edge intelligent agent inputs the above-mentioned control adjustment parameters into the proxy model in the form of vectors. The proxy model outputs the predicted values of several key production quality parameters caused in the downstream process equipment after the control parameters are applied to the current process equipment. In an embodiment of the present invention, the process equipment is a photolithography equipment, and the downstream process equipment is an etching equipment. The predicted values of the key production quality parameters include but are not limited to graphic transfer accuracy, etching depth uniformity or film thickness deviation.

[0072] To further quantify the sensitivity of these predicted values to downstream equipment performance, the rate of change of each predicted value relative to the input control parameter is differentiated at the output to obtain the corresponding impact gradient. This differential calculation is implemented using an automatic differentiation framework. For example, the finite difference method is used to apply a small perturbation to each control adjustment parameter. The change in the predicted output value is calculated and normalized to obtain the impact gradient. This impact gradient is used to measure the degree of quality fluctuation caused by the current adjustment strategy after execution. This mechanism helps to avoid the risk of overall process degradation caused by local optimization.

[0073] The edge agent of the process equipment compresses the impact gradient and sends it to the edge agent of the downstream process equipment through the edge communication channel.

[0074] The edge agent of the downstream process equipment introduces this influence gradient as a penalty factor in the decision-making process to make corrections, including:

[0075] After the edge agent of the downstream process equipment receives the influence gradient, it puts the influence gradient into the mapping set of influence gradient-penalty factor pre-stored in the database for mapping matching to obtain the penalty factor of the downstream process equipment, and introduces the penalty factor into the loss function of the strategy network parameters of the adjustment strategy model of the downstream process equipment for decision correction.

[0076] In an embodiment of the present invention, after the edge agent of the downstream process equipment receives the influence gradient from the upstream process equipment, in order to achieve coordinated regulation and quality risk control of the process chain, the following specific implementation steps are executed to complete the penalty factor introduction and policy model correction process: the edge agent uses the received influence gradient as input and searches for the preset influence gradient-penalty factor mapping set in the local or collaborative database. The mapping set is formed by a large number of historical regulation experiments and simulation data training, and describes the degree of influence of different influence gradient levels on the product quality stability of the downstream equipment. Through this mapping, a quantitative penalty factor is obtained, which is used to penalize the potential quality deviation caused by upstream parameter changes to the downstream. The penalty factor is introduced into the adjustment policy model of the current downstream equipment, specifically by embedding the loss function in the policy model. Taking the policy network as an example, the loss function is usually the expected gain loss in reinforcement learning, the action deviation loss, or the mean square error in supervised learning. At this time, the edge agent corrects the loss function to a combination with a penalty term, for example ,in, is the original loss term, is the penalty factor, To influence the gradient The penalty function, is the corrected loss item.

[0077] This decision correction mechanism enables the policy network to automatically select control actions that have less impact on downstream devices during training or inference, enabling flexible cross-device coordination. This ensures data-driven, coordinated control between upstream and downstream devices, significantly improving the overall quality coordination and dynamic adaptability of the complex chip manufacturing process.

[0078] The cloud update module is used to continuously update the local state quadruple to the cloud through the edge agent of the process equipment. The cloud identifies whether there is policy oscillation and energy consumption anomaly. If so, the process equipment is controlled to enter the cooling-off period lock mode.

[0079] The edge agent of the process equipment continuously updates the local state quadruple to the cloud, including:

[0080] The state quadruple includes the original data of device operation, the adjustment strategy model, the adjustment feedback data and the impact gradient. The parameters in the original data of device operation, the adjustment strategy model and the adjustment feedback data are numbered respectively to obtain the parameter set of the original data of device operation, the parameter set of the adjustment strategy model and the parameter set of the adjustment feedback data.

[0081] In an embodiment of the present invention, using lithography equipment as an example, adjustment feedback data represents the actual control effect parameters fed back by the process equipment after executing the strategy model. Specifically, it includes process indicator response delay and feedback offset. The process indicator response delay includes exposure time response delay, light source power adjustment response delay, and intensity filter adjustment response delay. Feedback offsets include exposure time offset, light source power offset, and filter stability offset. These parameters are measured and acquired via local closed-loop control logic or a feedback interface. Exposure time response delay refers to the time it takes for the actual exposure duration adjustment to take effect after the strategy is issued to the equipment's exposure unit. Light source power adjustment response delay refers to the time it takes for the equipment's light source output to reach the target power after the light source power adjustment command is issued. Intensity filter adjustment response delay refers to the time it takes for the light intensity signal collected by the sensor to stabilize in the new filtering state after the filter coefficient is adjusted. Exposure time offset refers to the difference between the target exposure time and the actual exposure time. Light source power offset refers to the difference between the target light source power and the actual output power. Filter stability offset refers to the deviation between the intensity fluctuation of the actual filtering result and the expected filtering result.

[0082] The state quadruple is compared with the parameter threshold set preset in the database. The parameter threshold set includes the equipment operation raw data threshold set, the adjustment strategy model threshold set, the adjustment feedback data threshold set and the impact gradient threshold. The equipment operation raw data threshold set includes the process parameter data threshold set, the electrical signal data threshold set and the environmental parameter data threshold set.

[0083] After obtaining the deviation value, weighted coupling is performed to obtain the state quadruple deviation evaluation value, which specifically includes:

[0084] ;

[0085] in, is the state quadruple deviation evaluation value, The first parameter set for the device to run the raw data The value of the parameter, The first parameter set for the device to run the raw data The threshold value of the parameter, The first parameter set for the device to run the raw data The weighting factors of the parameters satisfy , To adjust the policy model parameter set The value of the parameter, To adjust the policy model parameter set The threshold value of the parameter, To adjust the policy model parameter set The weighting factors of the parameters satisfy , To adjust the feedback data parameter set The value of the parameter, To adjust the feedback data parameter set The threshold value of the parameter, To adjust the feedback data parameter set The weighting factors of the parameters satisfy , To influence the gradient, To affect the gradient threshold, Weighting factors for equipment operation raw data, To adjust the strategy model weighting factors, To adjust the feedback data weighting factor, To influence the gradient weighting factor, The number of the parameter in the device operation raw data parameter set. , is the total number of parameters in the device operation original data parameter set, fl is the number of the parameter in the adjustment strategy model parameter set, and f , To adjust the total number of parameters in the policy model parameter set, To adjust the number of parameters in the feedback data parameter set, , To adjust the total number of parameters in the feedback data parameter set.

[0086] It should be noted that the weighting factor of the device operation original data, the weighting factor of the adjustment strategy model, the weighting factor of the adjustment feedback data and the weighting factor of the impact gradient meet the requirements in the embodiment of the present invention. When calculating the state quadruple deviation evaluation value, the weighting factor of the equipment operation original data is used to reflect the relative importance of the original operation data such as process parameters, electrical signals and environmental parameters in the overall deviation evaluation, ensuring the accurate representation of the actual operation status of the equipment; the adjustment strategy model weighting factor is used to measure the contribution of the strategy model parameters to the deviation evaluation, and by giving higher weights to key strategy parameters, the sensitivity to the strategy adjustment effect is improved; the adjustment feedback data weighting factor reflects the weight of feedback information in the deviation evaluation, emphasizing the attention to the actual adjustment effect and feedback response, which helps to timely discover anomalies or deviations in strategy execution; the influence gradient weighting factor is used to adjust the weight of the influence gradient on downstream equipment, reflecting the contribution of different influencing factors to the overall system state deviation, thereby supporting more comprehensive state evaluation and anomaly detection.

[0087] It should also be noted that, in the embodiment of the present invention, taking the lithography equipment as an example, the original data of the equipment operation includes process parameter data, electrical signal data and environmental parameter data, and the internal parameter correlation has been explained above.

[0088] The adjustment strategy model consists of a set of policy network parameters and a set of control adjustment parameters. The policy network parameters include the exposure time adjustment ratio, light source power correction ratio, and light intensity stabilization filter coefficient. The control adjustment parameters include the exposure time adjustment value, light source power adjustment value, and filter coefficient adjustment value. These parameters are closely related. Specifically, in this embodiment of the present invention, the policy network parameters are adjustment directional indicators obtained through learning and reasoning by the edge agent, specifically including the exposure time adjustment ratio, light source power correction ratio, and light intensity stabilization filter coefficient. They guide the device in which direction to adjust under current operating conditions to optimize operating conditions or maintain system stability. The control adjustment parameters are the specific numerical execution results of these policy directional indicators, including the exposure time adjustment value, light source power adjustment value, and filter coefficient adjustment value. They are calculated based on the current device's actual operating data (such as the current exposure time and light source power) and the proportional parameters provided by the policy network, and are used to drive the actual hardware to execute the adjustment instructions. Specifically, the exposure time adjustment value is calculated by multiplying the current exposure time by the exposure time adjustment ratio. For example, if the current exposure time is 5 seconds and the adjustment ratio is +0.12, the adjustment value is +0.6 seconds. Similarly, the light source power adjustment value is calculated based on the current light source power and the correction ratio. The filter coefficient adjustment value is directly involved in the signal processing step, used to control the device's response sensitivity to changes in light intensity. This conversion process from strategy ratio to execution value reflects a closed-loop structure of strategy guidance and control execution.

[0089] In an embodiment of the present invention, the adjustment feedback data, taking the lithography equipment as an example, includes process indicator response delay and feedback offset. There is a close correlation between the process indicator response delay and the feedback offset, which is manifested as causality, synergy and dynamic coupling. Generally speaking, the greater the response delay, the higher the possibility and degree of feedback offset. This is because the device may enter the next round of strategy adjustment before the response is completed, resulting in over-adjustment or hysteresis compensation. In addition, if the response delay shows volatility in different cycles, this instability will directly amplify the fluctuation of the feedback offset, making it difficult for the strategy model to be frequently revised but to converge stably, resulting in strategy oscillation. Therefore, response delay and feedback offset are not only two independent feedback data indicators, but also important factors that interact with each other in a dynamic system and jointly determine the effect of adjustment strategy execution and the stability of equipment energy consumption.

[0090] There is also a certain correlation between the major data categories of equipment operation raw data, adjustment strategy models, adjustment feedback data, and impact gradients. In the embodiments of the present invention, equipment operation raw data serves as the perception foundation for the entire system, primarily including process parameters, electrical signals, and environmental data. This data reflects the current operating status of the equipment and the process execution environment, and serves as the input for the adjustment strategy model. Based on the raw data input, the adjustment strategy model generates a set of control adjustment parameters and strategy network parameters, which specifically determine adjustments such as exposure time, light source power, and filter coefficients. This model continuously updates to seek the optimal control path to adapt to environmental fluctuations and changes in process objectives. After the parameters output by the strategy model are applied to the equipment, the system collects adjustment feedback data in real time, primarily including response delays and offsets of process indicators. This feedback data reflects the effectiveness of the current strategy model in actual execution, providing direct evidence for its modification and optimization. By pairing feedback data with raw data, the system can further assess whether the strategy adjustment has achieved its intended goals. Based on this feedback, the system further calculates the impact gradient, which is the sensitivity of a particular strategy parameter adjustment to the performance of downstream process equipment. This gradient information is not only used to evaluate the impact of the current policy, but also to control the policy's convergence speed and prevent policy oscillation. Ultimately, these four types of data continuously interact and optimize in a closed-loop manner. Raw device operation data drives policy adjustments, while the policy model acts on the device to generate feedback data. This feedback data is analyzed to generate an impact gradient, which in turn acts on the policy model to form a new adjustment strategy.

[0091] If the state quadruple deviation assessment value is greater than or equal to the deviation assessment threshold preset in the database, an emergency upload is triggered, the state quadruple is encapsulated into a structured data packet, and the edge upload channel is selected and sent directly to the cloud.

[0092] If the state quadruple deviation assessment value is less than the deviation assessment threshold, the edge agent of the process equipment will encapsulate the state quadruple into a structured data packet and upload it in batches based on the preset upload interval period in the database.

[0093] The cloud identifies whether there are policy fluctuations and abnormal energy consumption, including:

[0094] After the cloud receives the structured data packet of the status quadruple of the process equipment, it will give priority to parsing the structured data packet uploaded urgently, and perform a difference processing on the deviation assessment value of the urgently uploaded status quadruple and the deviation assessment threshold to obtain the status quadruple deviation difference, which is compared with the deviation difference threshold pre-stored in the database. If the status quadruple deviation difference is greater than or equal to the deviation difference threshold, it is determined that there is an abnormal fluctuation in the process equipment and an alarm is issued.

[0095] If the state quadruple deviation difference is less than the fluctuation deviation difference threshold, it is determined that there is no fluctuation abnormality in the process equipment, and the urgently uploaded structured data packet is marked and sent back to the edge agent for local agent model training.

[0096] The cloud parses the structured data packets of the status quadruple uploaded in scheduled batches and performs time alignment. When parsing each structured data packet, the cloud reads the upload timestamp and device unique identifier carried in it, sorts all uploaded data packets of the same device in ascending order by timestamp, and constructs an ordered state sequence.

[0097] Get the change value of each data in the policy network parameters at each upload interval, including:

[0098] ;

[0099] in, is the change value of the u-th data in the strategy network parameter during the t-th upload interval period, is the value of the u-th data in the strategic network parameter at the t-th upload interval period, is the value of the u-th data in the strategy network parameter at the t-1th upload interval period, is the policy network parameter data number, , is the total number of policy network parameter data, is the upload interval period number, , The total number of upload interval cycles.

[0100] After being modified and coupled with the corresponding unit weighted factor, the change range of the strategic network parameters of each upload interval period is obtained, including:

[0101] ;

[0102] in, is the change range of the strategic network parameters in the tth upload interval period, is the change value of the u-th data in the strategy network parameter during the t-th upload interval period, is the unit weighting factor corresponding to the u-th data in the network parameter, satisfying , u is the policy network parameter data number, , is the total number of policy network parameter data, is the upload interval period number, , The total number of upload interval cycles.

[0103] The change range of the strategic network parameters in each upload interval period is compared with the strategic network parameter change range threshold pre-stored in the cloud database, and the strategic network parameters whose change range is greater than or equal to the strategic network parameter change range threshold are recorded as significantly changed parameters.

[0104] Count the upload intervals during which parameters change significantly. If significant parameter changes occur in adjacent upload intervals, these intervals are counted as consecutive intervals. The number of consecutive intervals is calculated. If the number of consecutive intervals exceeds the threshold, the edge agent in the process equipment is considered to be experiencing policy oscillation. Policy oscillation refers to frequent and significant changes in the parameters in the policy model over a period of time. This is manifested as significant fluctuations in the policy network parameters over multiple consecutive upload intervals. This essentially reflects the failure of the edge agent to converge stably on a specific optimization strategy. This may be due to external interference, data anomalies, or a failure of the feedback mechanism, resulting in frequent policy updates with no significant effect. Policy oscillation generally indicates reduced system stability.

[0105] When there is policy oscillation in the edge intelligent body of the process equipment, the original data of the equipment operation in the state quadruple in the continuous cycle is retrieved, and the energy consumption data of the process equipment at each upload interval period in the continuous cycle is calculated in Python+engineering library based on the original data of the equipment operation. In an embodiment of the present invention, the power consumption value is used as the energy consumption data.

[0106] The energy consumption change value of each upload interval period in the continuous cycle is obtained by subtracting the energy consumption data of the adjacent upload interval period in the continuous cycle, and the energy consumption data change amplitude value in the continuous cycle is obtained after weighted coupling average processing, specifically including:

[0107] ;

[0108] in, is the change amplitude of energy consumption data in continuous cycles, The first Energy consumption change value of each upload interval period, The first The weight factor of the upload interval period satisfies , The upload interval period number in the continuous cycle. , The total number of upload intervals in a continuous cycle.

[0109] If the energy consumption data variation range exceeds the normal variation range preset for the equipment, it is determined that the process equipment has energy consumption anomalies.

[0110] Control the process equipment to enter the cool-off period lock mode. The specific processing conditions are:

[0111] The continuous cycle number is subtracted from the continuous cycle threshold to obtain the continuous cycle difference value, and the energy consumption data change amplitude value is subtracted from the normal change amplitude value to obtain the energy consumption change difference value.

[0112] Multiply the difference between consecutive cycles by the unit factor of the cycle quantity to eliminate the unit, multiply the difference between the energy consumption change and the unit factor of the energy consumption to eliminate the unit, and then add them together to obtain the cooling-off period length pointing factor. The cooling-off period length pointing factor is put into the mapping set of cooling-off period length pointing factor-cooling-off period length pre-stored in the cloud database for mapping and matching to obtain the cooling-off period length of the process equipment.

[0113] Based on the length of the cooling-off period, the process equipment is controlled to enter the cooling-off period lock mode. Specifically, the cooling-off period lock mode is that the adjustment strategy model of the edge intelligent body remains frozen, the process equipment maintains the existing state and caches all data in the edge intelligent body, and the cloud sends the process equipment number to the operation and maintenance management terminal for reminder.

[0114] like Figure 8 The figure shows the cooling-off period control page of the chip manufacturing operation and maintenance system involved in an embodiment of the present invention, which is used by administrators to view and manage the cooling-off period, including information such as the display of devices in the cooling-off period, the operation control interactive page, etc.

[0115] It also includes an emergency upload module, which is used to select the edge upload channel when an emergency upload is triggered. Specifically, it includes:

[0116] Obtain real-time performance data for each edge upload channel, including general indicator data and extended indicator data. Extract the hard thresholds of the general indicator data from the database. After comparison, select the edge upload channels that meet all hard thresholds and record them as general upload channels.

[0117] It should be noted that general indicator data include bandwidth occupancy, latency, packet loss rate and throughput; extended indicator data include median data transmission delay, number of retransmissions and bandwidth fluctuation value. These parameters are automatically collected and calculated by the network performance monitoring module built into the edge intelligent body.

[0118] The extended indicator data of each universal upload channel is compared with the set of extended indicator data standard values extracted from the database, including the standard value of the median data transmission delay, the standard value of the number of retransmissions, and the standard value of bandwidth fluctuation. After comparison, weighted coupling is performed to obtain the extended indicator value of each universal upload channel, specifically including:

[0119] ;

[0120] in, For the The extended index value of a general upload channel, For the The median data transmission delay of a common upload channel, For the The number of retransmissions for a general upload channel, For the The bandwidth fluctuation value of a general upload channel, is the median standard value of data transmission delay, is the standard value of the number of retransmissions, is the standard value of bandwidth fluctuation, is the weighting factor of the median data transmission delay, is the weighting factor for the number of retransmissions, is the bandwidth fluctuation weighting factor, is the number of the general upload channel, , is the total number of general upload channels.

[0121] It should be noted that the median data transmission delay weighting factor, the retransmission number weighting factor and the bandwidth fluctuation weighting factor all have a value range between 0 and 1 and meet the following requirements: The median data transmission delay weighting factor reflects the system's emphasis on channel transmission delay stability. The retransmission count weighting factor reflects the system's emphasis on data transmission reliability. This factor is used to measure the impact of retransmissions caused by network instability, interference, or packet loss. If the system has extremely high data integrity requirements, this factor should be set higher. The bandwidth fluctuation weighting factor represents the system's sensitivity to bandwidth stability. When the mission requires continuous high-bandwidth transmission, this weighting should be increased to ensure that the evaluation process places greater emphasis on the impact of bandwidth fluctuations on channel availability. These three weighting factors are generally preset based on the characteristics of the system's operating mission. Their sum is normalized to 1 and serves as the basis for weighted fusion calculation of the channel expansion performance index. The final channel expansion index score is obtained by summing the products of the three indicators and their corresponding weighting factors, reflecting the overall performance level of the channel.

[0122] It's also important to note that there's a correlation between median data transmission latency, retransmission count, and bandwidth fluctuation. Median data transmission latency reflects the typical delay level during packet transmission and is a key parameter for assessing network responsiveness. When packet loss or errors occur during network transmission, the system must retransmit. An increase in retransmissions significantly increases overall transmission latency, thereby increasing median latency. Therefore, there's a direct positive correlation between retransmission count and median latency. Bandwidth fluctuation measures the stability of available channel bandwidth. Large fluctuations indicate network congestion or jitter during certain periods, preventing the stable transmission of bursty data. Unstable bandwidth can lead to sudden bandwidth drops, preventing some packets from being transmitted in time or being lost, triggering retransmissions. Therefore, increased bandwidth fluctuations often lead to an increase in retransmission counts. Furthermore, bandwidth fluctuations can cause inconsistent transmission rates, exacerbating delays in packet queuing and waiting, leading to an increase in median latency. These three metrics form a dynamic feedback loop: increased bandwidth fluctuations can lead to increased retransmissions, which in turn increases the median data transmission latency. Together, these three metrics reflect the volatility and uncertainty of the upload channel. Therefore, when evaluating edge upload channel performance, these three metrics should be used as a linked evaluation system, weighted and coupled to comprehensively assess channel reliability and stability.

[0123] The general upload channel with the largest expansion index value is selected as the emergency upload channel.

[0124] like Figure 2 As shown, in this embodiment, the present invention provides a chip design and manufacturing optimization method based on edge reinforcement learning, including:

[0125] The original data of equipment operation is collected on the process equipment of chip manufacturing, and the state feature extraction and scene classification are performed to obtain the scene labels of the process equipment.

[0126] After the scene recognition is completed, the edge agent of the process equipment selects the corresponding policy model from the local policy library as the adjustment policy model according to the scene label.

[0127] The edge agent of the process equipment calculates the impact gradient of the adjustment strategy model on the downstream process equipment. At the same time, the edge agent of the downstream process equipment introduces the impact gradient as a penalty factor in the decision-making process to make decision corrections.

[0128] The edge agent of the process equipment continuously updates the local state quadruple to the cloud. The cloud identifies whether there are policy fluctuations and energy consumption anomalies. If so, the process equipment is controlled to enter the cooling-off period lock mode.

[0129] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0130] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all details in detail, nor do they limit the invention to only the specific embodiments described. Obviously, many modifications and changes can be made based on the contents of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can better understand and utilize the present invention. As long as they do not deviate from the structure of the present invention or exceed the scope defined by the present invention, they should fall within the scope of protection of the present invention.

Claims

1. A chip design and manufacturing optimization system based on edge reinforcement learning, characterized by: include: The scene recognition module is used to collect raw data from chip manufacturing process equipment, extract state features, and classify scenes to obtain scene labels for the process equipment; The strategy selection module is used to select the corresponding strategy model from the local strategy library as the adjustment strategy model based on the scenario label through the edge agent of the process equipment after the scenario recognition is completed; The decision correction module is used to calculate the impact gradient of the adjustment strategy model on the downstream process equipment through the edge agent of the process equipment. At the same time, the edge agent of the downstream process equipment introduces the impact gradient as a penalty factor in the decision-making process to make decision corrections; The cloud update module is used to continuously update the local state quadruple to the cloud through the edge agent of the process equipment. The cloud identifies whether there is policy oscillation and energy consumption anomaly. If so, the process equipment is controlled to enter the cool-down period lock mode; The edge agent of the downstream process equipment introduces the influence gradient as a penalty factor in the decision-making process to make decision corrections, specifically including: After receiving the influence gradient, the edge agent of the downstream process equipment puts the influence gradient into the mapping set of influence gradient-penalty factor pre-stored in the database for mapping matching to obtain the penalty factor of the downstream process equipment. The penalty factor is then introduced into the loss function of the policy network parameters of the adjustment policy model of the downstream process equipment to make decision corrections. The state quadruple includes the original data of equipment operation, the adjustment strategy model, the adjustment feedback data and the impact gradient; The process equipment is controlled to enter the cool-down period lock mode, and the specific processing conditions are: The continuous cycle number is subtracted from the continuous cycle threshold to obtain the continuous cycle difference value, and the energy consumption data change amplitude value is subtracted from the normal change amplitude value to obtain the energy consumption change difference value; The continuous cycle difference and the energy consumption change difference are respectively combined with the cycle quantity unit factor and the energy consumption unit factor, and then added to obtain the cooling-off period length pointing factor. The cooling-off period length pointing factor is input into the cooling-off period length pointing factor-cooling-off period length mapping set pre-stored in the cloud database for mapping and matching to obtain the cooling-off period length of the process equipment; Based on the length of the cooling-off period, the process equipment is controlled to enter the cooling-off period lock mode. Specifically, the cooling-off period lock mode is that the adjustment strategy model of the edge intelligent body remains frozen, the process equipment maintains the existing state and caches all data in the edge intelligent body, and the cloud sends the process equipment number to the operation and maintenance management terminal for reminder.

2. The chip design and manufacturing optimization system based on edge reinforcement learning according to claim 1 is characterized by: The process of collecting raw data of equipment operation on chip manufacturing process equipment, extracting state features and classifying scenes to obtain scene labels of process equipment is as follows: The equipment operation raw data includes process parameter data, electrical signal data and environmental parameter data; Extract the model of the process equipment and match it with the mapping set of equipment model-state perception feature channel pre-stored in the database to obtain the state perception feature channel of the process equipment. Input the original data of the equipment operation into the state perception feature channel of the process equipment to extract the state feature and obtain the labels of each preparatory scene of the process equipment. Extract the equipment operation reference data of each preparatory scenario label, including process parameter reference data, electrical signal reference data, and environmental parameter reference data, compare them with the original equipment operation data to obtain the equipment operation data deviation value set, and then perform weighted coupling to obtain the confidence score value of each preparatory scenario label; Compare the confidence score of each pre-construction scene label with the confidence score threshold pre-stored in the database, count the pre-construction scene labels with confidence scores greater than or equal to the confidence score threshold as the confidence scene labels, and select the confidence scene label with the largest confidence score as the scene label of the process equipment; If there are two or more confidence scene labels with the largest confidence score, the fuzzy recognition processing mode is executed, and these confidence scene labels are recorded as fuzzy scene labels; If the confidence score values of all preparatory scene labels are less than the confidence score threshold, an alarm will be issued.

3. The chip design and manufacturing optimization system based on edge reinforcement learning according to claim 1 is characterized by: The specific process of selecting a corresponding policy model from the local policy library as the adjustment policy model is as follows: After receiving the scene label, the edge agent on the process equipment establishes a secondary index anchor of the process equipment model-scene label and puts it into the local policy library for indexing. It obtains the policy model corresponding to the secondary index anchor and loads it as the adjustment policy model of the process equipment. The policy model includes policy network parameters and control adjustment parameters. If the fuzzy recognition processing mode is executed, the fuzzy index anchor points of the process equipment model-fuzzy scene label are established and put into the local policy library for indexing to obtain the policy models corresponding to the fuzzy index anchor points. The policy models are loaded and averaged to obtain the adjustment policy model of the process equipment.

4. The chip design and manufacturing optimization system based on edge reinforcement learning according to claim 1 is characterized by: The edge agent of the process equipment calculates and adjusts the impact gradient of the strategy model on the downstream process equipment, specifically including: The edge agent of the process equipment calls the local proxy model and inputs the control adjustment parameters of the adjustment strategy model. The proxy model outputs the predicted impact value of the control adjustment parameters on the production quality parameters of the downstream process equipment after application. The predicted impact value is differentiated to obtain the impact gradient on the downstream process equipment. The edge agent of the process equipment compresses the impact gradient and sends it to the edge agent of the downstream process equipment through the edge communication channel.

5. The chip design and manufacturing optimization system based on edge reinforcement learning according to claim 1 is characterized by: The edge agent of the process equipment continuously updates the local state quadruple to the cloud, specifically including: The state quadruple is compared with the parameter threshold set preset in the database. The parameter threshold set includes the device operation raw data threshold set, the adjustment strategy model threshold set, the adjustment feedback data threshold set, and the impact gradient threshold. After obtaining the deviation value, weighted coupling is performed to obtain the state quadruple deviation assessment value. If the state quadruple deviation assessment value is greater than or equal to the deviation assessment threshold, an emergency upload is triggered. If the state quadruple deviation assessment value is less than the deviation assessment threshold, the edge agent of the process equipment will encapsulate the state quadruple into a structured data packet and upload it in batches based on the preset upload interval.

6. The chip design and manufacturing optimization system based on edge reinforcement learning according to claim 1 is characterized by: The cloud-side identification of whether there is policy fluctuation and energy consumption anomaly specifically includes: After receiving the structured data packet of the process equipment's status quadruple, the cloud prioritizes parsing the urgently uploaded structured data packet, subtracts the urgently uploaded status quadruple deviation assessment value from the deviation assessment threshold to obtain the status quadruple deviation difference, and compares it with the deviation difference threshold pre-stored in the database. If the status quadruple deviation difference is greater than or equal to the deviation difference threshold, it is determined that the process equipment has an abnormal fluctuation and an alarm is issued; If the state quadruple deviation difference is less than the fluctuation deviation difference threshold, it is determined that there is no fluctuation abnormality in the process equipment, and the urgently uploaded structured data packet is marked and sent back to the edge agent for local agent model training; The cloud parses the structured data packets of the status quadruple uploaded in scheduled batches and performs time alignment to obtain the change values of each data in the policy network parameters in each upload interval period. After correction with the corresponding unit weighting factor, the change amplitude of the policy network parameters in each upload interval period is obtained and compared with the policy network parameter change amplitude threshold pre-stored in the cloud database. The policy network parameters with a change amplitude greater than or equal to the policy network parameter change amplitude threshold are recorded as significantly changed parameters. Count the upload intervals with significant parameter changes. If there are also significant parameter changes in adjacent upload intervals, they are recorded as consecutive periods. The number of consecutive periods is obtained. If the number of consecutive periods exceeds the consecutive period threshold, it is determined that the edge agent of the process equipment has policy oscillation. When there is policy oscillation in the edge intelligent body of the process equipment, the original data of the equipment operation in the state quadruple in the continuous cycle is retrieved, and the energy consumption data of the process equipment in each upload interval period in the continuous cycle is calculated based on the original data of the equipment operation. The energy consumption change value of each upload interval period in the continuous cycle is obtained by subtracting it from the energy consumption data of the adjacent upload interval periods in the continuous cycle. After weighted coupling averaging processing, the energy consumption data change amplitude value in the continuous cycle is obtained. If the energy consumption data change amplitude value exceeds the normal change amplitude value preset for the equipment, it is determined that there is energy consumption abnormality in the process equipment.

7. The chip design and manufacturing optimization system based on edge reinforcement learning according to claim 5 is characterized by: It also includes an emergency upload module, which is used to select the edge upload channel when an emergency upload is triggered. Specifically, it includes: Obtain real-time performance data for each edge upload channel, including general and extended indicator data. Extract the hard thresholds for the general indicator data from the database, compare them, and select the edge upload channels that meet all the hard thresholds. These are recorded as general upload channels. The extended index data of each general upload channel is compared with the extended index data standard value set extracted from the database, and then weighted coupling is performed to obtain the extended index value of each general upload channel, and the general upload channel with the largest extended index value is selected as the emergency upload channel.

8. The method applied to the chip design and manufacturing optimization system based on edge reinforcement learning according to any one of claims 1 to 7, characterized in that: Collect raw data from chip manufacturing process equipment, extract state features, and classify scenes to obtain scene labels for the process equipment. After the scene recognition is completed, the edge agent of the process equipment selects the corresponding policy model from the local policy library as the adjustment policy model according to the scene label; The edge agent of the process equipment calculates the impact gradient of the adjustment strategy model on the downstream process equipment. At the same time, the edge agent of the downstream process equipment introduces this impact gradient as a penalty factor in the decision-making process to make decision corrections. The edge agent of the process equipment continuously updates the local state quadruple to the cloud. The cloud identifies whether there are policy fluctuations and energy consumption anomalies. If so, the process equipment is controlled to enter the cooling-off period lock mode.

Images