Crystal growth monitoring method and apparatus based on multi-modal data, and storage medium

Abstract

The application relates to a crystal growth monitoring method and device based on multi-modal data and a storage medium. The method comprises the following steps: acquiring multi-modal data of a target crystal in a first target time period; extracting a feature vector from the multi-modal data in the first target time period based on multi-modal data in a second target time period; inputting a learning model to obtain at least one abnormal type; calculating the matching degree of the feature vector and a stage threshold value; when the matching degree is greater than or equal to a preset value, determining the grade of the abnormal type according to the matching degree and sending a control instruction to a crystal growth furnace according to the grade until the matching degree is less than the preset value; wherein the stage threshold value is set according to the current growth stage of the target crystal; and the above steps are repeated until the crystal growth is completed. The multi-modal data of the crystal is acquired to improve the monitoring information dimension, the matching degree with the stage threshold value is checked after the learning model outputs the abnormal type, misjudgment can be effectively prevented, and the regulation and control instruction is sent according to the grade after the abnormality is confirmed, until the crystal is completed.

Description

Technical Field

[0001] This application relates to the field of crystal growth technology, and in particular to a crystal growth monitoring method, device and storage medium based on multimodal data. Background Technology

[0002] During crystal growth, it is usually necessary to observe the crystal manually and adjust the parameters to complete the crystal growth. However, continuous observation is not possible, and there are time intervals between each observation. If the parameters are not adjusted in time, the crystal may be scrapped.

[0003] To achieve automated crystal growth control, image data analysis can be used to monitor crystal growth. However, the information provided by image data is limited. Image data cannot be combined with the weight changes during the lithium niobate growth process, making it difficult to obtain key parameters such as the lithium niobate growth rate. In addition, interference factors such as dust in the high-temperature furnace or light refraction can introduce noise into the image, leading to misjudgment. Crystal growth control usually uses a uniform and fixed recognition threshold, without considering the abnormal characteristics exhibited by lithium niobate at different growth stages, resulting in poor adaptability to the lithium niobate growth process and low monitoring and recognition accuracy. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of this application is to provide a crystal growth monitoring method, device and storage medium based on multimodal data. This application improves the dimension of crystal growth monitoring information by acquiring multimodal data of crystals. Crystals have multiple growth stages. After the learning model outputs the abnormal type, a matching degree verification with the stage threshold is added, which can effectively prevent misjudgment. After confirming the abnormality, a control command is sent according to the level until the crystal completes growth, realizing full-cycle cyclic monitoring and control.

[0005] To address the aforementioned technical problems, this invention provides a crystal growth monitoring method based on multimodal data, comprising: Acquire multimodal data of the target crystal during the first target time period; Based on the multimodal data of the second target time period, feature vectors for characterizing the growth state of the target crystal are extracted from the multimodal data of the first target time period. The feature vector is input into the learning model to obtain at least one anomalous type of the target crystal; The matching degree between the feature vector and the stage threshold is calculated. When the matching degree is greater than or equal to a preset value, the level of the anomaly type is determined according to the matching degree, and a control command is sent to the crystal growth furnace according to the level of the anomaly type, until the matching degree is less than the preset value; wherein, the stage threshold is set according to the current growth stage of the target crystal. Repeat the above steps until the growth of the target crystal is complete.

[0006] In one feasible embodiment, acquiring the multimodal data of the target crystal within a first target time period includes simultaneously acquiring image data, weight data, and temperature field data of the target crystal within the first target time period.

[0007] In one feasible embodiment, the feature vector includes weight deviation, growth rate abrupt change rate, temperature gradient deviation, texture entropy, grayscale fluctuation frequency, and diameter deviation; the feature vector characterizing the growth state of the target crystal is extracted from the multimodal data within the first target time period based on the multimodal data of the second target time period, including: The weight data and growth rate of the target crystal during the first target time period are extracted from the weight data during the second target time period. The weight deviation is obtained by subtracting two adjacent weight data of the target crystal during the first target time period. The growth rate mutation rate is calculated based on the growth rate of the target crystal during the first and second target time periods. The temperature gradient of the target crystal during the first target time period is extracted from the temperature field data during the first target time period. The temperature field gradient deviation is calculated based on the temperature gradient and standard gradient of the target crystal during the first target time period. The texture entropy and grayscale fluctuation frequency during the first target time period are calculated based on the image data during the first target time period. The sampled radial coordinates of the target crystal are extracted from the image data during the first target time period. The diameter deviation is calculated based on the sampled radial coordinates of the target crystal and the radial coordinates of the standard isodiameter curve. The second target time period is set before the first target time period.

[0008] In one feasible embodiment, extracting the temperature gradient of the target crystal from the temperature field data of the first target time period includes obtaining the temperature values ​​of at least three locations of the target crystal at any time point within the first target time period, and calculating the temperature gradient of the first target time period based on the temperature values ​​of the at least three locations.

[0009] In one feasible embodiment, the anomaly types include twinning, growth stripe anomalies, diameter fluctuations, and crystal cracking; the step of inputting the feature vector into the learning model to obtain at least one anomaly type of the target crystal includes, When the texture entropy meets the first condition, the anomaly type is twinning; when the grayscale fluctuation frequency meets the second condition, the anomaly type is growth stripe anomaly; when the diameter deviation meets the third condition, the anomaly type is diameter fluctuation; when the weight deviation, the growth rate abrupt change rate, the temperature field gradient deviation, the texture entropy, the grayscale fluctuation frequency, and the diameter deviation all meet the fourth condition, the anomaly type is crystal cracking.

[0010] In a feasible embodiment, after inputting the feature vector into the learning model, the method further includes the learning model outputting a confidence score, and when the confidence score is greater than or equal to a first judgment threshold, using the current anomaly type and its level as training samples, and updating the learning model based on the training samples; The training samples include a first sample and a second sample. The confidence level of the first sample is greater than or equal to the first judgment threshold, the confidence level of the second sample is greater than or equal to the second judgment threshold, and the first judgment threshold is less than the second judgment threshold.

[0011] In one feasible embodiment, the target crystal is lithium niobate, and the learning model is pre-trained based on the PyTorch framework.

[0012] Accordingly, this application also relates to a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the steps of the crystal growth monitoring method based on multimodal data.

[0013] Accordingly, this application also relates to a computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the steps of the crystal growth monitoring method based on multimodal data.

[0014] Accordingly, this application also relates to a computer program product, including a computer program that, when executed by a processor, implements the steps of the crystal growth monitoring method based on multimodal data.

[0015] This application improves the dimensionality of crystal growth monitoring information by acquiring multimodal data of the crystal. The crystal has multiple growth stages. After the learning model outputs the abnormal type, a matching degree verification with the stage threshold is added, which can effectively prevent misjudgment. After confirming the abnormality, the control command is sent according to the level until the crystal completes the growth, realizing full-cycle cyclic monitoring and control.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0018] Figure 1 This is a flowchart illustrating the steps of a crystal growth monitoring method based on multimodal data in one embodiment of this application; Figure 2 This is a flowchart illustrating the steps of a crystal growth monitoring method based on multimodal data in another embodiment of this application; Figure 3 This is a schematic diagram of the stage thresholds corresponding to different growth stages of the crystal in this application; Figure 4 This is a schematic diagram of the computer device of the present invention. Detailed Implementation

[0019] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0020] It should be noted that when a component is said to be "fixed to" another component, it can be directly attached to the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] Gallium oxide single crystal identification technology is only designed for the dissociation characteristics and impurity defects of gallium oxide, and is not adapted to the unique anomalies of lithium niobate single crystals such as twinning, growth stripes, and diameter fluctuations in the constant diameter stage, resulting in low identification accuracy. Crystal growth methods mostly rely on single image data and do not combine key parameters such as weight changes (growth rate) and temperature field distribution during the growth process of lithium niobate, making them prone to misjudgment due to image noise (such as dust in high-temperature furnaces and light refraction). They mostly rely on manual observation of the crystal during the crystal growth process and adjustment of parameters to complete the crystal growth, but manual observation cannot be continuous, and each observation is spaced out for a period of time, which can easily lead to crystal scrap due to adjustment delays. Crystal growth methods use a uniform identification threshold and do not consider the differences in abnormal characteristics of different growth stages of lithium niobate (such as cracking in the shoulder stage and diameter fluctuations in the constant diameter stage), resulting in poor adaptability.

[0023] Reference Figure 1 This invention provides a crystal growth monitoring method based on multimodal data, including: Step S100: Obtain multimodal data of the target crystal within the first target time period; Step S200: Based on the multimodal data of the second target time period, extract the feature vector used to characterize the growth state of the target crystal from the multimodal data of the first target time period; Step S300: Input the feature vector into the learning model to obtain at least one anomalous type of the target crystal; Step S400: Calculate the matching degree between the feature vector and the stage threshold. When the matching degree is greater than or equal to the preset value, determine the level of the anomaly type based on the matching degree and send a control command to the crystal growth furnace according to the level of the anomaly type until the matching degree is less than the preset value. The stage threshold is set according to the current growth stage of the target crystal. Step S500: Repeat the above steps until the growth of the target crystal is complete.

[0024] The first target time period of this application can be understood as one cycle, and steps S100-S400 are completed within one cycle. The second target time period can be understood as another cycle, and the two cycles are of equal duration, which can be 60 seconds. That is to say, a complete monitoring (steps S100-S400) occurs within one cycle. The crystal has multiple growth stages, including the shoulder formation stage, the constant diameter stage, and the termination stage.

[0025] In one feasible embodiment, acquiring multimodal data of the target crystal within a first target time period includes simultaneously acquiring image data, weight data, and temperature field data of the target crystal within the first target time period.

[0026] Specifically, the crystal is grown in a growth furnace, and an observation window is provided on the side wall of the growth furnace. In step S100, image data, weight data, and temperature field data of the crystal are acquired simultaneously within one cycle. This application addresses the same anomaly type by using multiple dimensional features from the image, weight, and temperature field to jointly point to and confirm the same anomaly. By using multiple features in parallel to judge and jointly support the same anomaly result, it avoids misjudgment caused by interference from a single feature.

[0027] For example, image data of the crystal is acquired via a camera with a resolution of 1920×1080, a frame rate of 30fps, and an operating temperature of -20℃ to 800℃. The camera focuses on the solid-liquid interface and the crystal outline, acquiring one frame per second to obtain image data. The camera is placed inside the furnace, with its lens facing the observation window of the growth furnace. A high-temperature resistant optical glass enclosure is used to directly capture the solid-liquid interface region, covering the crystal shoulder and the isodiameter region. The solid-liquid interface refers to the boundary between the solid crystal and the liquid melt. The camera is used to focus and capture the shape of the solid-liquid interface and the edges of the crystal outline.

[0028] For example, the weight data of the crystal is collected by a weight sensor placed on the seed crystal rod outside the furnace. The weight sensor is used to collect the weight data of the crystal once every 100ms, calculate the difference between two adjacent data and the time difference, and obtain the growth rate (g / h). The range of the weight sensor is 50kg and the accuracy is 0.01g.

[0029] For example, temperature field data of the crystal is acquired using at least three K-type thermocouples. The positions of the three K-type thermocouples must be fixed, and the installation accuracy must be controlled within ±1mm; otherwise, the temperature gradient calculation will be distorted. The three K-type thermocouples are respectively arranged above the solid-liquid interface at positions of 10mm, 30mm, and 50mm along the vertical direction. The measurement accuracy of a single K-type thermocouple is ±1℃. Temperature data is acquired every 200ms for each K-type thermocouple to calculate the three-point temperature gradient (℃ / cm). K-type thermocouples have a wide temperature measurement range, capable of measuring from -200℃ to 1300℃, perfectly covering the single crystal furnace process range. K-type thermocouples have good stability, low drift over long-term use, and are suitable for high-temperature, long-term monitoring. The cost and signal of K-type thermocouples are mature, with standardized signals, strong anti-interference capabilities, and easy integration with industrial PLCs or data acquisition cards.

[0030] After data acquisition is completed in step S100, the acquired data is preprocessed before step S200. For image data: First, an adaptive median filtering algorithm is used for noise reduction, eliminating isolated noise points caused by dust in the furnace. Then, to address the uneven brightness issue in high-temperature scenarios, the CLAHE (Contrast-Limited Adaptive Histogram Equalization) algorithm is used to improve the clarity of the solid-liquid interface contour. Finally, based on threshold segmentation and edge detection (Canny algorithm), the crystal contour and solid-liquid interface region are extracted, and background interference is eliminated. For weight data: A moving average filter is used to eliminate outliers caused by sensor signal jumps; the growth rate is calculated (growth rate = (current acquired weight - previous acquired weight) / acquisition time difference). For temperature field data: A Kalman filter algorithm is used to reduce temperature data fluctuations; the temperature gradient is calculated (gradient 1 = (T1 - T2) / 20, gradient 2 = (T2 - T3) / 20), where T1, T2, and T3 are the temperatures at the three acquired points, and the average of gradient 1 and gradient 2 is taken as the final temperature field gradient.

[0031] In one feasible embodiment, the feature vector includes weight deviation, growth rate abrupt change rate, temperature gradient deviation, texture entropy, grayscale fluctuation frequency, and diameter deviation; based on the multimodal data of the second target time period, feature vectors characterizing the growth state of the target crystal are extracted from the multimodal data of the first target time period, including... Multiple weighing data points and growth rate of the target crystal are extracted from the weight data of the first target time period. Similarly, multiple weighing data points and growth rate of the target crystal are extracted from the weight data of the second target time period. The weight deviation is obtained by subtracting two adjacent weighing data points of the target crystal in the first target time period. The growth rate mutation rate is calculated based on the growth rate of the target crystal in the first and second target time periods. The temperature gradient of the target crystal in the first target time period is extracted from the temperature field data. The temperature field gradient deviation is calculated based on the temperature gradient and standard gradient of the target crystal in the first target time period. The texture entropy and grayscale fluctuation frequency of the first target time period are calculated based on the image data of the first target time period. The sampled radial coordinates of the target crystal are extracted from the image data of the first target time period. The diameter deviation is calculated based on the sampled radial coordinates of the target crystal and the radial coordinates of the standard isodiameter curve. The second target time period is set before the first target time period.

[0032] For example, for the weight data collected in the first target time period, the weight data includes the weighing data and growth rate of the first target time period. As mentioned above, the weight deviation in the first target time period is obtained by calculating the difference between two weighing data collected in the first target time period. The growth rate in the first target time period is obtained by dividing the weight deviation by the time difference between the two weighing data collections. Similarly, the calculation process for the weight deviation and growth rate corresponding to the second target time period is the same, and will not be repeated here. The growth rate mutation rate is obtained by subtracting the absolute value of the growth rate in the second target time period from the growth rate in the first target time period, and then comparing this absolute value with the growth rate in the second target time period. This ratio is the growth rate mutation rate. As mentioned above, the temperature gradient is calculated by calculating the temperature values ​​collected by three thermocouples. The temperature field gradient deviation is obtained by dividing the absolute value of the difference between the temperature gradient in the current time period and the standard temperature gradient by the standard temperature gradient.

[0033] For example, the texture entropy and grayscale fluctuation frequency of the first target time period are calculated based on the image data of the first target time period; the texture entropy is expressed by the following formula. ; in, Represents texture entropy; L represents the total number of gray levels in the image, which is usually taken as 256; This represents the probability of the i-th gray level appearing in the solid-liquid interface region.

[0034] The grayscale fluctuation frequency is expressed by the following formula. ; in Indicates the frequency of grayscale fluctuation; 1 represents the number of grayscale signal fluctuations within one period; Indicates the duration of the period. =60s.

[0035] The diameter deviation is expressed by the following formula. ; in Indicates diameter deviation; This represents the total number of sampling points on the target crystal profile; This represents the radial coordinate of the k-th point of the current target crystal profile; This represents the radial coordinate of the k-th point corresponding to the standard isochronous curve.

[0036] In one feasible embodiment, the above-mentioned 6-dimensional feature vector is obtained by normalizing the multi-dimensional feature vector, which is used to identify the unique anomalies of lithium niobate crystals.

[0037] In one feasible embodiment, extracting the temperature gradient of the target crystal from the temperature field data of the first target time period includes obtaining the temperature values ​​of at least three locations of the target crystal at any time point within the first target time period, and calculating the temperature gradient of the first target time period based on the temperature values ​​of the at least three locations.

[0038] In one feasible embodiment, the anomaly types include twinning, growth stripe anomalies, diameter fluctuations, and crystal cracking; the feature vector is input into the learning model to obtain at least one anomaly type of the target crystal, including... When the texture entropy meets the first condition, the anomaly type is twinning; when the grayscale fluctuation frequency meets the second condition, the anomaly type is growth stripe anomaly; when the diameter deviation meets the third condition, the anomaly type is diameter fluctuation; when the weight deviation, growth rate abrupt change rate, temperature field gradient deviation, texture entropy, grayscale fluctuation frequency, and diameter deviation all meet the fourth condition, the anomaly type is crystal cracking.

[0039] For example, twinning refers to a local mismatch in the lattice arrangement during the growth of lithium niobate single crystals, resulting in two slightly different crystalline regions within the same grain. Twinning leads to inhomogeneity in the optical properties of the crystal and is a key defect affecting the performance of lithium niobate devices. It must be identified and terminated in the early stages of growth. The first condition is that the texture entropy is greater than 0.8, meaning that when the texture entropy calculated above is greater than 0.8, the model outputs anomaly type as twinning. The second condition is that the grayscale fluctuation frequency is less than 0.5Hz, meaning that when the grayscale fluctuation frequency is less than 0.5Hz, the model outputs anomaly type as growth stripe anomaly. The third condition is that the diameter deviation is greater than 0.05mm, meaning that when the diameter deviation is greater than 0.05mm, the model outputs anomaly type as diameter fluctuation. The fourth condition is that the texture entropy is greater than 0.8, the grayscale fluctuation frequency is less than 0.5Hz, the diameter deviation is greater than 0.05mm, the growth rate abrupt change rate is greater than 10%, the weight deviation is greater than 0.05g, and the temperature gradient deviation is greater than... When all conditions are met, the model outputs anomaly type "crystal cracking". It's important to note that one or more of these anomaly types can occur, but when crystal cracking occurs, other anomaly types become irrelevant.

[0040] In one feasible embodiment, such as Figure 3 As shown, the stage thresholds correspond to different stages; in the equal-diameter stage, different feature vectors correspond to different stage thresholds. Step 1: List the measured feature values. Texture entropy: x1 = 0.96, corresponding to threshold Th1 = 0.8 Gray-scale fluctuation frequency: x2 = 0.55Hz, corresponding to threshold Th2 = 0.5Hz. Average profile deviation: x3 = 0.06 mm, corresponding to threshold Th3 = 0.05 mm Weight change: x4 = 0.06g, corresponding to threshold Th4 = 0.05g; Calculate the matching degree for each abnormal feature. If the matching degree is greater than 80%, assign a risk level to the abnormality type. Twin matching degree: M1=0.96 / 0.8=1.2; Stripe matching degree: M2=0.55 / 0.5=1.1; Diameter matching degree: M3=0.06 / 0.05=1.2; Crack matching degree: M4 = 1 / 2 * (0.06 / 0.05 + 0.96 / 0.8) = 1.2; If the matching degree of the above four anomalies is satisfied, the highest risk item will be selected according to the weight priority, with the weight order as follows: cracking > twinning > diameter fluctuation > growth stripes. In this example, cracking has the highest weight.

[0041] For the above-mentioned anomalies that account for more than 80%, risk levels are classified as follows: minor anomalies (single feature exceeding the standard, with a matching degree of 80%-85%), moderate anomalies (two or more features exceeding the standard, or a matching degree of 85%-90%), and severe anomalies (three or more features exceeding the standard, or a matching degree of ≥90%). The alarm methods are as follows: minor anomalies (green indicator light + low-frequency buzzer (1 time / second)), moderate anomalies (yellow indicator light + medium-frequency buzzer (2 times / second)), and severe anomalies (red indicator light + high-frequency buzzer (3 times / second)).

[0042] In some embodiments, determining the level of the anomaly type based on the matching degree and sending control commands to the crystal growth furnace according to the level of the anomaly type includes, At Level 1, a heating command is sent to raise the temperature of the crystal growth furnace to remove the abnormal type and continue crystal growth. Specifically, this includes: first increasing the power to eliminate abnormal crystal components such as twins and growth streaks, then maintaining the current pulling speed and continuing the crystal growth program; increasing the monitoring frequency of corresponding out-of-range features (30s / cycle); and not triggering a shutdown, only recording abnormal data for model iteration. Level 1 represents a minor abnormality.

[0043] At the second level, a heating command is sent to raise the temperature of the crystal growth furnace to remove the abnormal type, and then the temperature field gradient is calibrated. Specifically, this includes: first increasing the power to eliminate abnormal crystal components such as twins and growth streaks, then adjusting the heating power and calibrating the temperature field gradient; moderately reducing the pulling speed (10%-15%); comprehensively increasing the monitoring frequency of all features (30s / cycle); pausing model iteration until the anomaly is alleviated. The second level represents a moderate anomaly.

[0044] At level three, a termination command is sent. This includes: immediately stopping crystal pulling and cutting off the heating power; automatically saving abnormal data and process parameters; locking the control module, requiring manual troubleshooting and reset before restarting. Level three represents a severe abnormality.

[0045] In a feasible embodiment, after inputting the feature vector into the learning model, the method further includes the learning model outputting a confidence score, and when the confidence score is greater than or equal to a first judgment threshold, using the current anomaly type and its level as training samples, and updating the learning model based on the training samples. The training samples include a first sample and a second sample. The confidence level of the first sample is greater than or equal to the first judgment threshold, and the confidence level of the second sample is greater than or equal to the second judgment threshold. The first judgment threshold is less than the second judgment threshold.

[0046] Reference Figure 2 Before acquiring multimodal data of the target crystal within the first target time period, the method further includes inputting initial growth parameters into the crystal growth furnace, where the target crystal grows. That is, system initialization is included before step S100. After initialization, multimodal data is acquired, preprocessed, and multi-dimensional feature vectors of the multimodal data are extracted. These feature vectors are then input into a learning model, which outputs the anomaly type and corresponding anomaly level, along with the corresponding confidence score. Specific anomaly verification and confidence score classification are as described above and will not be repeated here.

[0047] The first sample is one with a confidence level between 80% and 95%. This first sample is added to the training set after manual verification. The second sample is one with a confidence level greater than 95%. This second sample can be directly added to the training set so that the model can be updated in real time. If the confidence level is less than 80%, the sample is discarded.

[0048] For example, the confidence score is calculated by a lightweight CNN-Transformer hybrid model using a Softmax (6 encoder layers + 3 decoder layers) normalization function. This score is used to quantify the model's confidence level in the current anomaly classification result. The model outputs predicted scores for five states: normal, twinned, growth stripes, diameter fluctuation, and crystal crack. The Softmax function maps the scores to a probability distribution between 0 and 1, and the maximum probability value is the confidence score of the current classification.

[0049] The confidence level is calculated using the following formula. ; This represents the raw score of the i-th class output by the model; Represents the probability of the i-th class ( (within the range of 0 to 1) n represents the total number of categories, and n can be 5; e represents a natural number.

[0050] For example, suppose the model outputs 5 scores: Normal: 0.1; Twin: 5.2; Stripes: 0.3; Diameter Fluctuation: 0.2; Crack: 0.2. After Softmax calculation: Normal: 0.02; Twin: 0.91 (maximum); Stripes: 0.03; Diameter Fluctuation: 0.02; Crack: 0.02; Total = 1. Therefore, the confidence level for twinning is determined to be 0.91 (91%). In one feasible embodiment, the target crystal is lithium niobate, and the learning model is pre-trained based on the PyTorch framework. The learning model is initially trained using a lithium niobate-specific defect dataset, containing 500 normal samples, 150 twin samples, 150 samples of abnormal growth stripes, 100 samples of diameter fluctuations, and 100 samples of cracks, totaling 1000 samples. The anomaly type and level are manually labeled. The initial model is trained based on the PyTorch framework, achieving an initial recognition accuracy ≥90%. An online incremental update mechanism filters samples during runtime. When the model output confidence level is ≥95%, samples are automatically added to the incremental training set; when the confidence level is 80% ≤ confidence level <95%, manual review and labeling are triggered before adding samples to the training set; samples with a confidence level <80% are discarded. Every 10 valid samples are accumulated, a model parameter update is triggered using a mini-batch gradient descent algorithm (batch size = 5, learning rate = 0.001), with an update time ≤1 second, without interrupting the growth process. The output results are the anomaly type (twin / growth stripe anomaly / diameter fluctuation / cracking), anomaly level (slight / moderate / severe), and confidence level (0-100%).

[0051] The lithium niobate in the lithium niobate growth monitoring method of this application is the low-power short-wavelength visible light band heterogeneous integrated optoelectronic material in the BG2025034 Low-power Short-wavelength Visible Light Band Heterogeneous Integrated Optoelectronic Materials and Devices Research Project.

[0052] In the description of this specification, the references to "one embodiment," "an embodiment," and / or "some embodiments," "some embodiments," "other embodiments," "ideal embodiments," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative descriptions of the above terms do not necessarily refer to the same embodiment or example; certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; however, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0053] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

[0054] The basic concepts have been described herein. It is obvious that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this specification by those skilled in the art. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0055] Furthermore, unless expressly stated in the claims, the order of elements and sequences, the use of numbers and letters, or other names in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on an existing server or mobile device.

[0056] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0057] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are sometimes modified by the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters are taken into account a specified number of significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0058] In one exemplary embodiment, such as Figure 4A computer device is provided, which can be a terminal. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a crystal growth monitoring method based on multimodal data. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0059] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0060] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0061] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0062] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A crystal growth monitoring method based on multimodal data, characterized in that, include, Acquire multimodal data of the target crystal during the first target time period; Based on the multimodal data of the second target time period, feature vectors for characterizing the growth state of the target crystal are extracted from the multimodal data of the first target time period. The feature vector is input into the learning model to obtain at least one anomalous type of the target crystal; The matching degree between the feature vector and the stage threshold is calculated. When the matching degree is greater than or equal to a preset value, the level of the anomaly type is determined according to the matching degree, and a control command is sent to the crystal growth furnace according to the level of the anomaly type, until the matching degree is less than the preset value; wherein, the stage threshold is set according to the current growth stage of the target crystal. Repeat the above steps until the growth of the target crystal is complete.

2. The crystal growth monitoring method based on multimodal data according to claim 1, characterized in that, The acquisition of multimodal data of the target crystal within a first target time period includes simultaneously acquiring image data, weight data, and temperature field data of the target crystal within the first target time period.

3. The crystal growth monitoring method based on multimodal data according to claim 1 or 2, characterized in that, The feature vector includes weight deviation, growth rate abrupt change rate, temperature gradient deviation, texture entropy, grayscale fluctuation frequency, and diameter deviation; the feature vector characterizing the growth state of the target crystal is extracted from the multimodal data within the first target time period based on the multimodal data of the second target time period, including... The weight data and growth rate of the target crystal during the first target time period are extracted from the weight data during the second target time period. The weight deviation is obtained by subtracting two adjacent weight data of the target crystal during the first target time period. The growth rate mutation rate is calculated based on the growth rate of the target crystal during the first and second target time periods. The temperature gradient of the target crystal during the first target time period is extracted from the temperature field data during the first target time period. The temperature field gradient deviation is calculated based on the temperature gradient and standard gradient of the target crystal during the first target time period. The texture entropy and grayscale fluctuation frequency during the first target time period are calculated based on the image data during the first target time period. The sampled radial coordinates of the target crystal are extracted from the image data during the first target time period. The diameter deviation is calculated based on the sampled radial coordinates of the target crystal and the radial coordinates of the standard isodiameter curve. The second target time period is set before the first target time period.

4. The crystal growth monitoring method based on multimodal data according to claim 3, characterized in that, Extracting the temperature gradient of the target crystal from the temperature field data of the first target time period includes obtaining the temperature values ​​of the target crystal at at least three locations at any time point within the first target time period, and calculating the temperature gradient of the first target time period based on the temperature values ​​at the at least three locations.

5. The crystal growth monitoring method based on multimodal data according to claim 3, characterized in that, The anomaly types include twinning, abnormal growth stripes, diameter fluctuations, and crystal cracking; the feature vector is input into the learning model to obtain at least one anomaly type of the target crystal, including... When the texture entropy meets the first condition, the anomaly type is twinning; when the grayscale fluctuation frequency meets the second condition, the anomaly type is growth stripe anomaly; when the diameter deviation meets the third condition, the anomaly type is diameter fluctuation; when the weight deviation, the growth rate abrupt change rate, the temperature field gradient deviation, the texture entropy, the grayscale fluctuation frequency, and the diameter deviation all meet the fourth condition, the anomaly type is crystal cracking.

6. The crystal growth monitoring method based on multimodal data according to claim 1, characterized in that, After inputting the feature vector into the learning model, the method further includes the learning model outputting a confidence score. When the confidence score is greater than or equal to a first judgment threshold, the current anomaly type and its level are used as training samples, and the learning model is updated based on the training samples. The training samples include a first sample and a second sample. The confidence level of the first sample is greater than or equal to the first judgment threshold, the confidence level of the second sample is greater than or equal to the second judgment threshold, and the first judgment threshold is less than the second judgment threshold.

7. The crystal growth monitoring method based on multimodal data according to claim 1, characterized in that, The target crystal is lithium niobate, and the learning model is pre-trained based on the PyTorch framework.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor runs the computer program stored in the memory, the processor performs the steps of the crystal growth monitoring method based on multimodal data as described in any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it is used to implement the steps of the crystal growth monitoring method based on multimodal data as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the crystal growth monitoring method based on multimodal data as described in any one of claims 1-7.

Images