Diamond synthesis system and method using machine learning

Abstract

Disclosed herein are systems and methods for synthesizing diamonds using a diamond synthesis apparatus. A processor receives multiple images of a diamond being synthesized within the diamond synthesis apparatus, each of the multiple images being captured within a time period. The processor runs a diamond condition prediction machine learning model using the multiple images to obtain a prediction data object, the prediction data object indicating the predicted state of the diamond within the diamond synthesis apparatus at a later time within the time period. The processor detects predicted defects, the number of defects, the type of defects, and / or quasi-characteristics of such defects, and / or other characteristics of the predicted state of the diamond (e.g., predicted shape, size, and / or other properties of the predicted outline of the diamond and / or pocket holder). The processor coordinates operation of the diamond synthesis apparatus.

Description

Technical Field

[0001] (Reference to Related Applications) This application claims priority based on U.S. Provisional Patent Application No. 63 / 029,177, filed on May 22, 2020, which is incorporated herein by reference for all purposes.

[0002] The present disclosure generally relates to artificial intelligence-based algorithms, and more particularly, to machine learning techniques for separating unique characteristics and predicting diamond crystal growth states for diamond synthesis and any by-products of such crystal growth (e.g., polycrystalline diamond growth).

Background Art

[0003] The chemical vapor deposition (CVD) method for diamond is a generally recognized manufacturing method for producing rough stones for gemstones and diamonds for electronic devices, optical devices, and quantum devices, and is therefore the most applicable method for mass production. However, due to the slow development of large-scale diamond synthesis, it has not been able to reach the commercial base easily. Generally, CVD is also a technique used for the mass synthesis of semiconductor materials such as polycrystalline Si for the solar cell market, and SiC and GaN for electronics. As of 2018, the sales of the semiconductor market exceeded $480 billion across industries such as home appliances, communications, defense, and computing. In the CVD systems for these materials, it is necessary to control a number of time-dependent process parameters of the growing crystal.

[0004] Current methods for manufacturing and producing diamonds using diamond synthesis rely on a reactive "guess and check" approach. Engineers can grow diamonds using a diamond synthesis apparatus and evaluate the diamonds after they have been placed in the apparatus for a predetermined time or after they have grown to a predetermined size. Engineers can search for defects in the diamonds, record their findings to indicate whether there are defects, different characteristics of the defects, and the settings used by the diamond synthesis apparatus to grow the diamonds, which resulted in the defects in the diamonds. In some cases, engineers can also determine whether the diamonds are of the desired shape and / or size. For each defect, incorrect size, or shape, the engineers can set the apparatus to new settings in the next attempt to grow diamonds without any defects and with the correct size and shape. Since it can take weeks to grow a single diamond, this reactive approach can require a lot of manual labor to test various operating parameters and ultimately find the parameters that will result in diamonds without any defects and with the correct size and shape. Additionally, since there are many diamond properties that can affect the presence or absence of diamond defects, it is difficult to consider every permutation of diamond properties when growing diamonds, and the optimal operating parameters for synthesizing one diamond may be completely different from those for another diamond. Summary of the Invention Problems to be Solved by the Invention

[0005] For the reasons described above, an automated technique can be used to avoid defects from occurring within the diamond before they occur, and as a result, a system is desired that can avoid the use of guesswork and verification methods that often result in waste of material when technicians try to find the correct parameters for diamond synthesis. Technicians may try to observe the diamond while it is being synthesized, but considering that the size of defects that may appear in the diamond is often small and the time when defects may appear during diamond synthesis cannot be predicted, it can be difficult to predict whether the diamond being synthesized currently has defects, let alone whether it will have any defects in the synthesis process later. Further, technicians may wish to control or adjust the operating parameters so that the diamond synthesis apparatus can synthesize a diamond of a desired size (or a desired amount of crystallization) by being able to predict the final dimensions of the diamond (e.g., the lateral dimensions) and the crystallization that may occur when the diamond is synthesized. Therefore, the system is required to automatically adjust the operating parameters of the diamond synthesis apparatus in real time so as to predict whether the diamond to be synthesized may have defects or whether the diamond may have a predetermined size or amount of crystallization, and prevent the diamond from having defects or an undesirable size or amount of crystallization.

Means for Solving the Problems

[0006] To overcome these technical deficiencies, it is desirable to use a series of device machine learning models to capture time-series images of diamonds (e.g., optical images or thermal images) during synthesis by a diamond synthesis device and predict how the diamonds will grow during future synthesis time steps. The system can execute a machine learning model using a series of images of the diamond captured during synthesis as input data and predict an image of the diamond at one or more time steps ahead. Since the growth of the diamond may depend on the state of the diamond at the previous time step (e.g., the size and shape of the diamond in another predicted image), a machine learning model (e.g., a machine learning model implementing sequence learning such as a network architecture consisting of units for a long short-term memory network) can predict the final state of the diamond in the predicted image or data object based on the predicted image or data object of the diamond at the previously predicted time step. The system can evaluate the final predicted image or data object to determine whether there are defects or other problems (e.g., diamonds of undesirable dimensions) in the predicted image or data object, and after determining that the current operating parameters result in the synthesis of defective diamonds or diamonds of undesirable dimensions, it is possible to adjust or specify the optimal operating parameters (e.g., temperature and / or pressure) of the diamond synthesis device. In some cases, after the system determines that the diamond seed during synthesis may result in defective or near-optimal-sized diamonds, the operator can first replace the diamond placed in the diamond synthesis device for synthesis with another diamond seed. By the system performing this process periodically while synthesizing the diamond until an image of a defect-free diamond is predicted, a guided diamond synthesis system can be realized.

[0007] With a guided or automated system, a diamond synthesis apparatus can synthesize diamonds of any size and / or any shape, including diamonds with characteristics that have not yet been evaluated, which is considered particularly useful when growing large diamonds that are prone to defects. Thus, the systems and methods provided herein completely avoid the speculation and confirmation methods for identifying appropriate parameters for diamond synthesis. Instead, the diamond synthesis apparatus uses a guided approach that can reduce the number of defects that appear in the diamond, create defect-free diamonds, synthesize diamonds of optimal size and / or shape, and / or optimize the amount of polycrystalline diamond that grows with the diamond during synthesis.

[0008] In one embodiment, a method of synthesizing diamonds using a diamond synthesis apparatus includes receiving, by a processor, a plurality of images of a diamond being synthesized within the diamond synthesis apparatus, each of the plurality of images being captured within a period; executing, by the processor, a diamond state prediction machine learning model using the plurality of images to obtain a prediction data object, the prediction data object indicating a predicted state of the diamond within the diamond synthesis apparatus at a time after the period; detecting, by the processor, predicted defects of the diamond based on the predicted state of the diamond; and adjusting, by the processor, an operation of the diamond synthesis apparatus in response to the detection of the defects.

[0009] In another embodiment, a system for synthesizing diamond using a diamond synthesis apparatus includes a processor configured to execute instructions stored on a non-transitory computer-readable medium to receive a plurality of images of a diamond during synthesis within the diamond synthesis apparatus, each of the plurality of images being captured within a predetermined period, execute a diamond state prediction machine learning model using the plurality of images to obtain a prediction data object indicative of a predicted state of the diamond within the diamond synthesis apparatus at a time after the period, detect a predicted defect within the diamond based on the predicted state of the diamond, and adjust the operation of the diamond synthesis apparatus in response to the detection of the defect.

[0010] In a further embodiment, the processor may be further configured to detect shape and / or size characteristics of the synthesized diamond. The processor may be configured to determine the shape and / or size of the diamond based on the contour of the prediction data object (e.g., the contour of the prediction image data object) output by a segmentation machine learning model that processes the prediction data object. The processor may be configured to compare the contour to a set of criteria and, when it determines that the contour meets at least one of the criteria, adjust the operation of the diamond synthesis apparatus based on the at least one met criteria.

[0011] In yet another embodiment, a method for detecting defects in a diamond comprises receiving, by a processor, a plurality of images of the diamond during synthesis in a diamond synthesis apparatus, each of the plurality of images being captured within a period; executing, by the processor, a diamond state prediction machine learning model using the plurality of images to obtain a prediction data object, the prediction data object indicating a predicted state of the diamond in the diamond synthesis apparatus at a time after that period, the diamond state prediction machine learning model being trained based on a time series of images corresponding to a set of images of previously synthesized diamonds; detecting, by the processor, predicted defects in the diamond based on the predicted state of the diamond; and transmitting, by the processor in response to the detection of the defect, a signal including an indication of the detected defect to a second processor.

[0012] Non-limiting embodiments of the present disclosure are illustrated by reference to the accompanying figures, which are schematic and not intended to be drawn to scale. The figures represent aspects of the present disclosure unless shown as representing the background art.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Best Mode for Carrying Out the Invention

[0014] Next, referring to the illustrated exemplary embodiments, where the same thing will be described using a specific language. Nevertheless, it will be understood that the scope of the claims or the disclosure is not thereby intended to be limited. Changes and further modifications of the features of the invention shown in this specification, as well as additional applications of the principles of the subject matter shown in this specification that would occur to those of ordinary skill in the relevant art and those of ordinary skill in the art possessing this disclosure, should be considered within the scope of the subject matter disclosed in this specification. Other embodiments may be used and / or other changes may be made without departing from the spirit or scope of the present disclosure. The exemplary embodiments described in the detailed description are not meant to limit the subject matter presented in this specification.

[0015] Current methods for manufacturing and producing diamonds using diamond synthesis methods rely on reactive speculation and confirmation methods. Considering the large number of parameters that can be input into a diamond synthesis apparatus to synthesize diamonds and the fact that the growth pattern of diamonds when growing from diamond seeds generally cannot be predicted, it is very difficult to determine which operating parameters to use to synthesize diamonds and whether to make adjustments to optimize the growth pattern of diamonds. Instead, engineers often input a set of parameters that are considered to make the fully synthesized diamond "close enough" and rely on their own knowledge to determine when to stop the diamond synthesis. Since diamond synthesis can take several weeks, it can also require thousands of man-hours to determine the operating parameters used to synthesize diamonds so that they are in a generally desirable state (size, shape, number of defects, etc.). Furthermore, even if the system determines parameters that allow it to generate the desired diamond from a diamond seed with specific characteristics, since the growth of the diamond during synthesis is a probabilistic process, engineers may have to repeat the synthesis operation for diamond seeds with other characteristics. Since the permutations of diamond characteristics are enormous, there is a need for a system that can synthesize diamonds in real-time considering characteristics that the system has not yet encountered in order to save the time, cost, and raw materials that would be spent on experiments to find the optimal operating parameters for synthesis.

[0016] By implementing the systems and methods described herein, the system can solve the aforementioned diamond synthesis defects by eliminating the need to use guess-and-verify techniques. Instead, the system can use the time-series image data obtained during synthesis and a series of machine learning models to predict how diamonds grow at various times during the synthesis period. The system can predict the state of the diamond at one or more time steps and use information from previously predicted time steps to predict the state of the diamond at the next time step using a machine learning model (e.g., an architecture having components similar to a network architecture consisting of units for long short-term memory networks, variations thereof, and other alternatives). The system can evaluate the predicted growth state of the diamond at each time step during the synthesis period to determine whether it is necessary to adjust the operating parameters of the diamond synthesis apparatus to improve the final diamond configuration (e.g., removing possible defects that may appear in the diamond at the end of the synthesis process, improving the shape and / or size characteristics of the diamond, increasing the amount of polycrystalline diamond formed alongside the diamond, etc.). If the system identifies a predicted diamond defect, the system can generate operating parameters and send them to the diamond synthesis apparatus, adjust its operation, and repeat this process until the system predicts that the final diamond is defect-free or until the synthesis period ends.

[0017] Advantageously, by implementing the systems and methods described herein, the system can implement a guided method for synthesizing diamonds using a diamond synthesis apparatus to create diamonds with fewer defects and / or having more optimal shape and / or size characteristics. Further, this solution enables the synthesis of diamonds with fewer defects and more optimal characteristics, regardless of the characteristics of the diamond seeds of the diamonds or changes in the composition of the diamonds that occur during synthesis. Thus, the system can avoid testing the operating parameters each time the characteristics of the diamonds change while improving the quality of the synthetic diamonds produced by the diamond synthesis apparatus.

[0018] As described below, a server (referred to herein as an analysis server) can receive a time-series image of a diamond being synthesized in a diamond synthesis apparatus (e.g., a continuous image, optionally associated with a timestamp), and using machine learning techniques, can depict how the diamond will look, or show an image or another data object indicating the predicted state of the diamond after being synthesized for a predetermined period or number of time steps. The server can analyze the predicted image or data object to determine whether there are defects in the predicted diamond or the predicted state of the diamond, or whether the diamond will grow into an undesirable shape or size, and can make some adjustment to the diamond synthesis apparatus (e.g., change an operating parameter) that can help avoid the predicted defects or undesirable shape or size. In a non-limiting example, the server can acquire a time-series image of a diamond being synthesized within the diamond synthesis apparatus. The server can use the time-series image as input data to a machine learning model to obtain a predicted image or data object of the diamond for a predetermined number of time steps, and the machine learning model iteratively predicts the state of the diamond at each of the time steps, using the predicted value of the previous time step to predict the state of the diamond at the next time step. The server can evaluate the final image or data object using either a set of criteria or another machine learning model to determine whether the diamond has defects or will become an undesirable size or shape. If the server detects a defect or an undesirable size or shape in the depicted or predicted state of the diamond, it can send an instruction to change the operation of the apparatus to the diamond synthesis apparatus for the purpose of avoiding the defect or undesirable size or shape. The server may repeat this process until it obtains a prediction that the diamond will be defect-free and / or have a desirable shape and / or size upon completion, thereby enabling a method of guided diamond synthesis. Figure 1 is a non-limiting example of the components of a system in which the analysis server operates.

[0019] FIG. 1 is a diagram showing the components of a diamond synthesis system 100. The system 100 can include an analysis server 110a, a system database 110b, electronic data sources 120a-d (collectively electronic data sources 120), end-user devices 140a-c (collectively end-user devices 140), and an administrator computing device 150. The above-described components may be connected to each other via a network 130. Examples of the network 130 include, but are not limited to, a private or public LAN, WLAN, MAN, WAN, and the Internet. The network 130 can include wired and / or wireless communications according to one or more standards and / or via one or more transport media.

[0020] Communications via the network 130 can be performed according to various communication protocols such as the Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and IEEE communication protocols. In one example, the network 130 may include wireless communications according to the Bluetooth specification set or another standard or proprietary wireless communication protocol. In another example, the network 130 can also include communications on a cellular network, such as, for example, a GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), or EDGE (Enhanced Data for Global Evolution) network.

[0021] The components of the system 100 are not limited to the components described herein and can include additional or other components to be considered within the scope of the embodiments described herein that are not shown for the sake of brevity.

[0022] The analysis server 110a may generate and display an electronic platform configured to train and / or use various computer models (including artificial intelligence and / or machine learning models) to predict how a diamond grows during a period at predetermined time steps using a time-series image of the diamond during synthesis, and to predict whether the predicted diamond has defects and / or the shape or size of the predicted diamond. The electronic platform may include a graphical user interface (GUI) displayed on each electronic data source 120, end-user device 140, and / or administrator computing device 150. Examples of the electronic platform generated and managed by the analysis server 110a may include web-based applications or websites configured to be displayed on different electronic devices such as mobile and wearable devices, tablets, personal computers, etc. In a non-limiting example, a technician operating the technician-operated device 120b can train a machine learning model to predict a data object indicating the predicted state of the diamond based on a sequence of time-series images of the diamond captured during synthesis. To do so, the user can create a labeled dataset of a sequence of images of the diamond during synthesis within the diamond synthesis device, along with labels indicating the ground truth data object or an image of the state of the diamond at a future predetermined time step, to generate a training dataset. The technician-operated device can send the labeled dataset to the analysis server 110a and / or save the labeled dataset on the technician-operated device side. The technician-operated device 120b or the analysis server 110a can then supply the labeled dataset to one or more machine learning models configured to predict the state of the diamond during synthesis and perform training.Once sufficiently trained, the analysis server 110a uses the trained machine learning model to predict the state of the diamond being synthesized as a data object or an image, and supplies the data object or the image to another machine learning model to predict whether the predicted data object or image has defects and / or a desired size and / or shape after being synthesized within a predetermined period.

[0023] As described herein, the diamond synthesis parameters (or operating parameters) may be any parameter related to synthesizing or growing a diamond in a diamond synthesis apparatus, or may include the same. For the diamond synthesis parameters, temperature, pressure, input gas, input power, etc. can be used, but are not limited thereto. In some embodiments, the diamond synthesis parameters may vary depending on the type of diamond synthesis apparatus. For example, in a diamond processing machine configured to synthesize diamonds using only heat and pressure, the input gas cannot be used as a parameter. The diamond synthesis parameters may be any parameter, or may include the same.

[0024] The analysis server 110a may manage a website accessible to a user (e.g., an end user) who operates any of the electronic devices described herein, and the content presented via various web pages may be under management based on the role and / or viewing permission of each specific user. The analysis server 110a may be any computing device including a processor and a non-transitory device-readable storage device capable of executing various tasks and processes described herein. Non-limiting examples of such computing devices may include workstation computers, laptop computers, server computers, laptop computers, and the like. Although the system 100 includes a single analysis server 110a, the analysis server 110a may include any number of computing devices operating in a distributed computing environment such as a cloud environment.

[0025] The analysis server 110a may execute a software application configured to display an electronic platform (e.g., host a website), thereby generating and providing various web pages to each electronic data source 120 and / or end-user device 140. If different users use the website, it is possible to display the results predicted by the machine learning model and / or exchange information.

[0026] The analysis server 110a can be configured to request user authentication based on a series of user authentication credentials (e.g., username, password, biometric authentication, cryptographic certificate, etc.). The analysis server 110a can access a system database 110b set to store user credentials, and the analysis server 110a can be set to refer to the system database to determine whether the input series of credentials (to be authenticated as the user) matches the appropriate series of credentials for identifying and authenticating the user.

[0027] In addition, the analysis server 110a can store data related to each user who operates one or more electronic data sources 120 and / or end-user devices 140. The analysis server 110a can use the data to compare and consider interactions while training various artificial intelligence (AI) models accordingly. For example, the analysis server 110a can indicate that the user is a domain expert and that the user's input is monitored and can be used to train the machine learning or other computer models described herein.

[0028] The analysis server 110a can generate and manage a web page (e.g., a web page for a dashboard) based on the role of a specific user within the system 100. In such an implementation, the role of the user can be defined by the data fields and input fields of the user record stored in the system database 110b. The analysis server 110a may authenticate the user and may identify the role of the user by executing an access directory protocol (e.g., LDAP). The analysis server 110a may generate web page content customized according to the role of the user defined by the user record in the system database 110b.

[0029] The analysis server 110a can receive diamond synthesis data (e.g., an image of time-series diamond synthesis image data) from a user or obtain such data from a data repository, analyze the data, and display the results on an electronic platform. For example, in a non-limiting example, the analysis server 110a can retrieve a diamond synthesis image from the database 120d by querying. The analysis server 110a can analyze the acquired data using various models (stored in the system database 110b). Thereafter, the analysis server 110a can display the results on the electronic engineer device 120b, the end-user device 140, and / or the administrator computing device 150 via the electronic platform.

[0030] The electronic data source 120 can represent various electronic data sources that contain, obtain, and / or input data related to diamond synthesis parameters, captured images of diamonds during synthesis, and / or predicted images of diamonds during synthesis. For example, the analysis server 110a can use the laboratory computer 120a, the technician device 120b, the server 120c (related to the technician and / or the laboratory), and the database 120d (related to the technician and / or the laboratory) to obtain / receive diamond synthesis parameters to be transmitted to the diamond synthesis apparatus 160, and can adjust the manner in which the diamond synthesis apparatus 160 operates to synthesize diamonds.

[0031] The end-user device 140 can be any computing device equipped with a processor and a non-transitory device-readable storage device capable of performing various tasks and processes described herein. For example, the models and image processing steps described herein may be stored and executed by the end-user device 140. Non-limiting examples of the end-user device 140 may be a workstation computer, a laptop computer, a tablet computer, and a server computer. During operation, various users can use the end-user device 140 to access the GUI operated and managed by the analysis server 110a. Specifically, the end-user device 140 may be the laboratory computer 140a, the laboratory server 140b, and / or the laboratory device 140c, including mobile or wearable devices such as mobile phones, tablet computers, or digital glasses.

[0032] The administrator computing device 150 may refer to a computing device operated by a system administrator. The administrator computing device 150 can be configured to display data (e.g., various analysis metrics, defect identification, and / or diamond prediction images) obtained or generated by the analysis server 110a, which allows the system administrator to monitor various models utilized by the analysis server 110a, the electronic data source 120, and / or the end-user device 140, review feedback, and / or facilitate the training of machine learning models maintained by the analysis server 110a.

[0033] The diamond synthesis device 160 can include a press machine (e.g., a belt press, a cubic press, a split sphere (BARS) press), a chemical vapor deposition machine, a device using explosive technology, or a device using ultrasonic cavitation. In operation, the technician can place one or more diamonds (e.g., diamond seeds) within or on a diamond holder of the diamond synthesis device (e.g., the diamond synthesis device 160), select options on the user interface of the diamond synthesis device or another device, and input parameters for synthesizing the diamond. The user can input one-time parameters (e.g., set temperature and / or pressure values) that the device can maintain throughout operation, or a schedule of parameters indicating the time the device operates under different settings. In some examples, the user can select an identifier of a pre-set schedule from a list of one or more pre-set schedules. The pre-set schedule can include parameters that the diamond synthesis device uses to synthesize diamonds over time. After selecting the parameters or the schedule of parameters, the user can select the option to initialize the diamond synthesis process with the selected parameters or schedule.

[0034] Upon initialization, the diamond synthesis apparatus can operate to synthesize diamond based on the selected parameters. After a predetermined time, the diamond synthesis apparatus can start capturing an image of the diamond and transmitting the image to the analysis server 110a. In addition to, or instead of, this, the diamond synthesis apparatus may transmit to the analysis server 110a operation parameters indicating how it operates at various times while the diamond synthesis apparatus is synthesizing diamond (e.g., parameter input or selected schedule, etc.). On the other hand, the analysis server 110a uses the image and the systems and methods described herein to predict an image of the diamond (or another data object such as a value or vector) at a future point in time, and based on the current parameters and / or operation schedule, when synthesizing diamond, can detect whether defects will occur in the diamond, or whether an inappropriate shape or size will result upon completion of synthesis. Based on the detection of diamond defects or inappropriate shape or size, the analysis server 110a generates configuration data for controlling the diamond synthesis apparatus and transmits an instruction to the synthesis apparatus to change the way the diamond synthesis apparatus operates to synthesize diamond, thereby preventing the diamond synthesis apparatus from creating defective diamonds and / or enabling the diamond to have an optimal shape or size.

[0035] The diamond synthesis apparatus 160 can capture a time-series image of the diamond being synthesized with new parameters, transmit the image to the analysis server 110a, and repeat the process. The diamond synthesis apparatus 160 can repeat the process until it determines that the current schedule of parameters will result in a defect-free diamond and / or will have an optimal shape or size, or until the diamond is synthesized to a predetermined size or within a predetermined period.

[0036] FIG. 2 shows a flowchart of a method 200 that is executed in a diamond synthesis system according to one embodiment. Method 200 may include steps 210-240. However, in other embodiments, additional or alternative steps may be included or one or more steps may be completely omitted. Method 200 is described as being executed by a data processing system (e.g., an analysis server 110a, a data source 120, or a computer similar to the end-user device 140 described in FIG. 1). However, one or more steps of method 200 may be executed by any number of computing devices operating in the distributed computing system described in FIG. 1. For example, one or more computing devices may execute some or all of the steps described in FIG. 2 on their own systems, or a cloud device may execute such steps.

[0037] In step 210, the data processing system can receive a plurality of images of the diamond being synthesized within the diamond synthesis apparatus. The images may be two-dimensional images or three-dimensional images, and may optionally include pixels or voxels. As described herein, when the relevant image is a three-dimensional image, each reference to a pixel may be a reference to a voxel. Each image can depict a diamond located on a diamond holder (e.g., a metal container configured to hold the diamond at a predetermined position within the diamond synthesis apparatus during synthesis) while the diamond is being synthesized or grown within the diamond synthesis apparatus. The diamond synthesis apparatus can be configured to synthesize the diamond to a desired size or a predetermined size and / or for a predetermined period of time. The images may be captured or taken at predetermined time intervals by an image capture device (such as a camera, phone, recorder, etc.). The image capture device may be configured to capture images at predetermined time intervals and continuously transmit the captured images to the data processing system for processing. The image capture device can attach a time stamp to each image to indicate when the device captured the image, when the device transmitted the image to the data processing system, and / or when the data processing system received the image. The data processing system can receive each image, identify the time stamps of a predetermined number of images within a predetermined period or the images associated with the time stamps, compile the images into a sequence, input the sequence into a machine learning model, and further process it.

[0038] In step 220, the data processing system can execute a diamond state prediction machine learning model using a plurality of images and obtain a data object indicating the predicted state of the diamond at a future time step. The data object may be a predicted image of the diamond, a value, a multi-dimensional value, a text description, or other things representing the diamond. The machine learning model can be any machine learning model (e.g., a neural network-based model, a random forest, a support vector machine, etc.) configured to generate a data object indicating the predicted state of the diamond in a diamond synthesis apparatus at a certain future point in time. The data processing system can execute the diamond state prediction machine learning model by obtaining a plurality of images (e.g., an image sequence of the diamond during synthesis) from a storage device or memory and generating a feature vector representing the pixels of the images. The data processing system may apply the feature vector to the diamond state prediction machine learning model to obtain a data object indicating the predicted state of the diamond at a predetermined time step towards the future. For example, the diamond state prediction machine learning model can depict an image of a future state diamond indicating how the diamond has grown, or generate crystallization formed in the diamond synthesis apparatus during synthesis.

[0039] To generate a feature vector based on a plurality of images, the data processing system can generate a feature vector having values representing each pixel of the plurality of images. The values may be values representing the color, brightness, and other visual aspects of the plurality of images. In some cases, the data processing system can normalize the input values to values between -1 and 1 or between 0 and 1 in order to obtain more accurate results when the diamond state prediction machine learning model processes the feature vector.

[0040] The diamond state prediction machine learning model may be a deep learning model based on units for convolutional long short-term memory configured to predict the state of diamonds (e.g., size, shape, and / or any defects) within a diamond synthesis apparatus for multiple future time steps. The diamond state prediction machine learning model may be configured to generate a prediction data object of the diamond by sequentially predicting the state of the diamond for different future time steps, and in some cases, the prediction data object or the prediction data object of the diamond from a previous time step may be used to generate a prediction object for the next time step. For example, the diamond state prediction machine learning model may use a plurality of images as input data to predict the first state of the diamond two hours into the future in a prediction data object (e.g., a multi-dimensional vector having values representing the pixels of the predicted image or the predicted image itself). The diamond state prediction machine learning model may use the prediction data object of the diamond in the first state as input data to the diamond state prediction machine learning model to predict the second state of the diamond four hours into the future in another prediction data object such as a multi-dimensional vector or an image. The diamond state prediction machine learning model can continue to predict the state of the diamond until it reaches the last time step for which the diamond state prediction machine learning model is configured to make predictions. The diamond state prediction machine learning may be configured to use any number of time steps of any length to predict the final state of the diamond. By using a network architecture that includes units for long short-term memory networks, considering that the growth of the diamond depends on its immediately preceding state and that a slight variation at the beginning of the growth cycle can cause the diamond to have a wide range of potential growth patterns in the future, the machine learning model may be able to more accurately predict the shape and size of the diamond in the future.

[0041] In some cases, in addition to using a plurality of images as input data to the diamond state prediction machine learning model, in data processing, a schedule of operation parameters may be used as input data. The schedule of operation parameters may be a temporal schedule of parameters used by the diamond synthesizer to synthesize diamonds. The schedule can be input by a user or an administrator and can include schedule values corresponding to the time when each image was captured and / or the time between the time when the image was captured and the time of the prediction data object (this may help to account for changes in the operation of the diamond synthesizer that may affect diamond growth). For example, the diamond synthesizer can be scheduled to apply a relatively high pressure on the second day of the synthesis process and increase or decrease the pressure at a predetermined rate or to a predetermined value until the diamond synthesizer finishes synthesizing the diamond. The diamond synthesizer can perform similar scheduling for other operation parameters. The data processing system can obtain the operation parameters used by the diamond synthesizer to synthesize the diamonds depicted in the plurality of images and include the parameters, optionally together with corresponding timestamps, in the feature vector used by the data processing system to execute the diamond state prediction machine learning model. By considering the operation parameters of the diamond synthesizer during synthesis, the diamond state prediction machine learning model can more accurately predict the final state of the diamond. Further, in some cases, the training data set used to train the diamond state prediction machine learning model can be controlled to include a high amount of variance (potentially within the limits of safe operation) between the operation parameters and the diamond characteristics so that the diamond state prediction machine learning model can learn a generalized function for diamond growth.

[0042] To generate or train a diamond state prediction machine learning model, the data processing system may use one or more labeled training data sets for supervised training. For example, the data processing system may create or use a training data set of a time series of images corresponding to a set of images of previously synthesized diamonds (e.g., images capturing the growth process of previously synthesized diamonds). The time series of images may be images capturing the diamond during synthesis. When the diamond state prediction machine learning model predicts an image of a diamond at a future point in time. The data processing system may create a training data set from a series of images and obtain a labeled image of the diamond from an image capture device captured at a predetermined time after the series of images are captured. The predetermined time may be the time during which the diamond state prediction machine learning model is learning to predict an image of the synthesized diamond. The data processing system can input a series of images into the diamond state prediction machine learning model to obtain a predicted image depicting the diamond in the predicted state. The data processing system can compare the output predicted image with the labeled image, determine the difference between the two images, and adjust the internal weights and parameters of the machine learning model in proportion to the difference determined according to the loss function, thereby training the diamond state prediction machine learning model.

[0043] If a diamond state prediction machine learning model is configured to predict other data objects (e.g., values or feature vectors) that indicate the predicted state of a diamond during synthesis, the diamond state prediction machine learning can also be trained using one or more labeled training data sets for supervised training. For example, the data processing system may use a selected training data set consisting of a time series of images corresponding to a set of images of previously synthesized diamonds. Each image in the time series can be labeled with a ground truth value or a feature vector label that indicates the state of the diamond at a predetermined future time during synthesis. The data processing system may input a series of images into the diamond state prediction machine learning model and obtain a predicted data object that depicts the predicted state of the diamond. The data processing system may compare the output predicted data object with the ground truth data object to determine the difference between the data objects and adjust the internal weights and parameters of the machine learning model in proportion to the difference determined according to a loss function to train the diamond state prediction machine learning model.

[0044] The data processing system can train the diamond state prediction machine learning model in a similar manner until the diamond state prediction machine learning model is determined to be accurate up to a threshold. If the diamond state prediction machine learning model is determined to be accurate up to the threshold, the data processing system may implement a machine learning model for real-time data object prediction of the diamonds synthesized by the diamond synthesis apparatus. After implementing a data processing system for real-time data object prediction, the data processing system can continue to train the machine learning model using the data generated during synthesis, taking into account changes in the operation of the diamond synthesis apparatus that may occur over time.

[0045] In step 230, the data processing system can detect the predicted defects of the diamond based on the predicted data object. The predicted defects are likely to be blockages, chips, color defects, macroscopic defect contours, microscopic defect contours, central defects, edge defects, or other crystallographic defects. As described herein, the defects may include undesirable shapes or sizes, or undesirable amounts of polycrystalline diamond growth. The data processing system may use post-processing techniques on the predicted data object of the diamond to detect the defects. In some examples, when the data object is an image, the data processing system detects the defects by evaluating the diamonds depicted in the predicted image according to a series of diamond defect criteria. For example, the data processing system can identify the shape and / or size of the diamond based on the contour of the diamond depicted in the predicted image and compare the shape or size with a series of diamond defect criteria. The data processing system determines whether the shape or size meets the criteria and may determine that the diamond has a defect depending on the conclusion of whether the shape or size meets the criteria (e.g., the diamond is too small, too large, or has an undesirable shape). In another example, the data processing system can determine the size of the diamond based on the contour of the depicted diamond in the predicted image and compare the size with a criterion (e.g., a threshold). If the size meets the criterion, the data processing system can determine that the diamond has a defect. In yet another example, the data processing system may identify crystallization (e.g., polycrystallization) within the predicted image that is not part of the predicted diamond or part of the diamond holder. The data processing system identifies the crystallization based on the fact that the crystal is separated from the diamond and, in some cases, can determine that the diamond in the predicted image has a defect based on the predicted crystallization.In yet another example, the data processing system can compare the defect contours identified by the data processing system within the image, compare the defect contours to one or more criteria, and based on the contours that match the one or more criteria, determine that the diamond has one or more macro defect contours, micro defects, center defects, and / or edge defects. In yet another example, if the predicted data object is a value or vector, the data processing system can compare the value or vector to a criterion. If the value or vector matches the criterion, the data processing system can determine that the diamond has a defect. The data processing system can determine that the diamond has a defect based on any criterion.

[0046] In some cases, the data processing system may use a feature extraction machine learning model to determine whether the diamond shown in the predicted data object has defects and / or the type (e.g., inclusion, chip, color defect, macro defect contour, micro defect, center defect, edge defect, or any other crystallographic defect) if there are defects. If the predicted data object is an image, the feature extraction machine learning model may be configured to perform a multi-class prediction indicating whether individual pixels indicate diamond defects and / or the type of such defects. In such a case, the data processing system can generate a feature vector from the output data of the diamond state prediction machine learning model (e.g., a predicted image depicting the predicted state of the diamond) and apply the feature vector to the feature extraction machine learning model. When processing the feature vector, the feature extraction machine learning model can output a label for a pixel or portion of the image indicating whether the pixel indicates a defect and / or the type of such a defect. The data processing system can evaluate the output labels for individual pixels and determine whether there are pixels (and / or contours) labeled with defect and / or defect type labels, thereby determining whether there are defects in the predicted state of the diamond.

[0047] When the prediction data object is a value or a vector, the feature extraction machine learning model may be configured to perform a multi-class prediction indicating whether the value or vector indicates that the diamond has a defect in the predicted state corresponding to the value or vector. In such a case, the data processing system may generate a feature vector from the value or vector and apply the feature vector to the feature extraction machine learning model. When processing the feature vector, the feature extraction machine learning model can output an indication of any defects and / or defect types in the predicted state of the diamond corresponding to the value or vector.

[0048] In some cases, the data processing system may use the feature extraction machine learning model to determine the predicted characteristics of the predicted state of the diamond. For example, when the data object is a predicted image, instead of or in addition to predicting the defects and / or defect types of the diamond in the predicted state, the feature extraction machine learning model can also predict a label that identifies a cross-section of the diamond in the predicted state (e.g., diamond top, diamond side 1, diamond side 2, pocket holder, etc.). The data processing system may identify the labels of different pixels of the image and, based on the labels, determine the shape and / or contour of the diamond in the predicted state. For example, the data processing system can increment a counter for each pixel label and determine the size and / or shape of the diamond based on the incremented counter. Each pixel may correspond to a set size, and the data processing system may determine the number of pixels with the same label and multiply that number by the pixel size to determine the size of the diamond and / or the portion of the pocket holder. The data processing system may use the size of the diamond to determine whether there is a defect related to the size in the diamond, or otherwise, whether it is growing to an undesirable size or shape (e.g., by comparison with stored criteria for size and / or shape).

[0049] Similarly, in an example where the predicted data object is a value or a vector, the feature extraction machine learning model can be configured to predict the size and / or shape of the diamond in the predicted state based on the predicted value or vector. In such a case, the data processing system may generate a feature vector from the value or vector and apply this feature vector to the feature extraction machine learning model. When processing the feature vector, the feature extraction machine learning model outputs a display of the size and / or shape of the diamond in the predicted state of the diamond, and based on the output display, it can be determined whether the size and / or shape is an undesirable size or shape.

[0050] The feature extraction machine learning model can currently use a "fully convolutional" segmentation architecture that includes a combination of an encoder and a decoder to classify the received image. The encoder serves to downsample to a feature vector learned from a 5,000x2,400 image, and the decoder can reconstruct the segmentation mask of the diamond and the pocket holder by combining upsampling techniques. The architecture of the encoder can be composed of a 34-layer residual network consisting of a 7x7x64 convolutional filter, four 3x3x64 convolutional layers, eight 3x3x128 convolutional layers, and finally four 3x3x256 layers. This architecture can appropriately incorporate pooling layers and batch normalization layers to add regularization functions. The feature extraction machine learning model can calculate the log-likelihood using softmax activation in the output layer and generate a 256-dimensional probabilistic feature vector. The decoder can then fuse and upsample the feature maps from three pooling layers to reconstruct the segmentation mask of the target object. The feature extraction machine learning model can use an 8-fold upsampling technique to accurately measure the data distribution. Due to the non-linear learning function of the neural network / deep learning, it can capture and generalize the complex distribution pattern of the target image and predict with an accuracy of over 99% with the machine learning model for feature extraction.

[0051] In step 240, the data processing system can adjust the operation of the diamond synthesis apparatus in response to the detection of a defect or the diamond growing to an undesirable shape or size. The data processing system can adjust the operation of the diamond synthesis apparatus by sending an instruction including one or more operating parameters (e.g., pressure or temperature) and / or a parameter schedule for operating the diamond synthesis apparatus based on the transmitted parameters or the parameters to the diamond synthesis apparatus. The data processing system can select or obtain one or more parameters from the memory and send the selected parameters to the diamond synthesis apparatus in response to the detection of a defect or the diamond growing to an undesirable shape or size.

[0052] In some cases, the data processing system can select one or more parameters based on the detected defect, the type of defect, and / or the undesirable shape or size. For example, the data processing system can identify the type of defect as described above and use the defect type as a search means in the memory to determine the parameters corresponding to the detected defect type. The data processing system can identify the matching parameters or parameters corresponding to the defect type, obtain the identified parameters, and use them to adjust the operation of the diamond synthesis apparatus. Similarly, the data processing system can use the undesirable shape or size as a search means to select parameters for identifying the parameters.

[0053] The data processing system may additionally or alternatively generate a record indicating that a defect and / or an undesirable shape or size has been detected in the predicted image or data object, the type of defect, and / or the parameters that the data processing system has obtained to resolve or avoid the defect and / or the undesirable shape or size. The data processing system may send the record to a processor such as an end-user device for display on a user interface. The user can display the information to attempt to prevent the occurrence of defects, undesirable shapes, or sizes, or otherwise analyze the defects, shapes, or sizes so that similar problems do not occur in the future.

[0054] In some cases, the data processing system can execute the steps described above and determine that there are no defects in the predicted image or data object and / or that the diamond has the desired shape and / or size. In such a case, the data processing system can generate a record (e.g., a file, document, table, list, message, notification, etc.) indicating that no defects were found and / or that the diamond is predicted to have the desired shape and / or size, and transmit that record to the computing device and / or diamond synthesis device. The record can be displayed at the receiving device and can be used to instruct not to adjust the parameters of the diamond synthesis device. If the data processing system determines that the current operating parameter schedule enables the diamond synthesis device to synthesize diamonds that are defect-free and / or have the desired shape or size, the data processing system can stop evaluating the images transmitted to the data processing system by the image capture device or transmit a signal to the image capture device to stop the image capture device from capturing or transmitting such images. Thereby, the data processing system can conserve the energy resources required for capturing, transmitting, receiving, and / or storing the image. In an alternative embodiment, the data processing system does not transmit a signal to the image capture device, and instead, it is also possible to repeatedly repeat the above process considering unexpected growth patterns of the diamond that may occur during synthesis.

[0055] The data processing system may store in a storage device or memory the data generated by the machine learning model, as well as the analysis data generated when the data processing system analyzes the data to detect defects and / or undesirable shapes or sizes. The data processing system can store such data long-term so that the data can be compared while diamond synthesis is being repeated. In some cases, if a third party requests a guarantee that the produced diamonds are of high quality or requests the method by which the diamonds were made, the stored data can be used as evidence for claims such as quality and origin. If there is a request for the stored data, the data processing system can generate a record based on the request and send the record to the requester.

[0056] The data processing system can extend the processes described herein to increase the growth of parasitic polycrystalline diamonds during diamond synthesis. Such growth allows the diamond synthesis apparatus to promote the growth of diamond material from diamond seeds. For example, in addition to training a diamond state prediction machine learning model and a feature extraction machine learning model to accurately predict not only the growth state of the diamond but also the defects of the diamond predicted during synthesis and / or the undesirable sizes and / or shapes of the diamond, the data processing system can train a machine learning model to predict polycrystalline diamond growth and make adjustments to optimize the amount of polycrystalline diamond growth that can occur on diamond seeds during synthesis. The machine learning model may be trained to predict cases where the synthesized diamond does not produce sufficient polycrystals, and as described above, may be trained to predict cases where the synthesized diamond has defects or has an undesirable shape or size, and the data processing system can use a quasi-optimal prediction of polycrystalline diamond growth to adjust the parameters of the diamond synthesis machine of the guidance system. Thus, the systems and methods described herein can result in an increase in the yield of diamond seeds during synthesis.

[0057] In some cases, an operator can use the systems and methods described herein to confirm that the diamond seeds placed by the operator on a device can grow into diamonds of a desired size and shape. For example, assume that a diamond with dimensions of 1.5A×15.B is desired to be grown. The operator can place the diamond seed in the diamond synthesis apparatus and initiate diamond synthesis.

[0058] Referring now to FIG. 3, a flowchart of a method 300 for training a machine learning model to generate a predicted growth image of a diamond during synthesis is shown. Method 300 can include steps 304, 310, 312, 316, and / or 320. However, in other embodiments, additional or alternative steps can be included, or one or more steps can be completely omitted. Method 300 is described as being performed by user 302 and / or a data processing system (e.g., a computer similar to analysis server 110a, data source 120, or end-user device 140 described in FIG. 1). However, one or more steps of method 300 can be executed by any number of computing devices operating in the distributed computing system described in FIG. 1. For example, one or more computing devices can execute some or all of the steps described with respect to FIG. 3 on their own systems, or a cloud device can execute such steps.

[0059] User 302 may be a human operator such as a domain expert in process cycle development. The operator can identify and label features in the image and train a machine learning model based on the generated labeled training data. The operator may use the graphical user interface 306 to label time-series data for prescribing, which may include the data processing system predicting a growth state at a future point in time using a growth prediction model and subsequently analyzing the output predicted growth state using a segmentation model. These models can be integrated into the manufacturing equipment for informed decision-making (e.g., a closed-loop control system).

[0060] For example, in step 304, user 302 can access the graphical user interface 306 and label the image with labels indicating whether the pixels of the image depict diamonds, diamond holders, backgrounds, and / or possibly diamond defects and / or defect types in some cases. The user can do so by drawing a contour on the image to label the pixels. The user interface may be synchronized with a labeled time-series database 308 that stores labeled image data that can be used for training. By labeling the image data, user 302 can enable the data processing system to use a supervised training method to train a machine learning model for segmenting the image for feature extraction and for diamond growth prediction.

[0061] In step 310, data from the manufacturing data cloud synchronization pipeline may be uploaded to the labeled time-series database 308. The data may include off-site data from a data provider that contains information about diamonds and diamond defects. Such information can include the operating parameters used in diamond generation or growth, and any defects formed in the resulting grown diamond. Further, this information may optionally include labeled training images. The data processing system can receive this data and store it in the labeled time-series database 308. This data can be used for training machine learning models for feature extraction and predicting diamond growth, as well as for determining whether there are diamond defects in predicted images and data objects. Optionally, the data can be used to train a machine learning model to predict the size and / or shape of a diamond in a predicted state.

[0062] In step 312, the data processing system may train the segmentation model 314 to extract features from an image of a diamond on a diamond holder (e.g., a diamond within a diamond synthesis apparatus). The data processing system may additionally and / or alternatively train the segmentation model 314 or another model to detect defects or undesirable shapes and / or sizes in a diamond in an image (e.g., a predicted image) using labeled training data from the labeled time series database 308. The data processing system may similarly train the segmentation model 314 to predict defects and / or undesirable shapes and / or sizes based on other data objects representing the predicted state of the diamond during synthesis. Similarly, in step 316, the data processing system may train the growth prediction model 318 to generate a predicted image of the diamond (or another data object indicating the predicted state of the diamond) at a future predetermined time based on a series of images of the diamond during synthesis and / or the operating parameters of the diamond synthesis apparatus performing the synthesis. The output data of the segmentation model 314 and the growth prediction model 318 may be synchronized with the user interface 306 so that the user 302 can check the progress of the training of the models 314 and 318. When sufficiently trained, in step 320, the data processing system may export the trained machine learning models to different devices (e.g., a diamond synthesis apparatus, an analysis server, an end-user device, etc.) to assist with a guided diamond synthesis process as described herein.

[0063] Referring now to FIG. 4, a flowchart of a method 400 for synthesizing diamonds using a guided approach is shown. In method 400, a data processing system (e.g., a computer similar to analysis server 110a, data source 120, or end-user device 140 described in FIG. 1) can execute a diamond state prediction machine learning model 402 using a sequence of diamond images to generate a predicted image of the diamond (or another data object indicative of the predicted state of the diamond) during synthesis by a diamond synthesis apparatus. The data processing system can detect diamond defects and / or undesirable size or shape, adjust the configuration of the diamond synthesis apparatus in a closed-loop control system, and optimize the diamond output from the diamond synthesis apparatus.

[0064] For example, a user can initialize the diamond synthesis process by placing a diamond seed in a diamond synthesis apparatus, selecting parameters for synthesis, and selecting options for causing the diamond synthesis apparatus to synthesize diamonds. While the diamond synthesis apparatus is synthesizing diamonds, an image capture device can capture images of the diamonds at predetermined time intervals. The image capture device can transmit the images as on-site data 404 (e.g., in-device data) to a data processing system. The data processing system may collect reactor data 406 (e.g., scheduled operating parameters of the diamond synthesis apparatus for synthesizing diamonds), and off-site data 408. The off-site data 408 may include defect data generated outside the apparatus, and may also include criteria or other data that can be used to detect defects or undesirable shapes and / or sizes in a predicted image or data object indicating the predicted state of the diamond. The diamond state prediction machine learning model 402 can generate, at step 410, a predicted data object indicating the predicted state of the diamond at a future time. The data processing system evaluates the data object and, at step 412, based on the predicted state of the diamond, detects that the diamond has a defect or that the diamond does not have a desired shape or size, and at step 414, can determine the parameters to transmit to the diamond synthesis apparatus. The data processing system, at step 416, adjusts the diamond synthesis apparatus and may repeat the process of method 400 until the diamond synthesis process is complete or until the data processing system determines that the diamond state prediction machine learning model 402 has output a predicted data object of a defect-free diamond and / or determines that an indicator is output that the diamond has been synthesized to have a desired shape or size (e.g., a predetermined shape or size, optionally within a buffer range).

[0065] Referring now to FIG. 5, there is shown a diagram of an image sequence of the predicted labels for the image of the diamond being synthesized and the predicted labels for the image. Through the user interface, a human operator can label the contour features on the objects in the images of the image sequence 502 to specify "ground truth" and create a ground truth image sequence 504. The image sequence 502 may be an image of the diamond in the diamond synthesis apparatus captured continuously at predetermined intervals, or may include the same. The user can view the images of the image sequence 502 and use the contour tool to specify the pixels of the image sequence 502 or label them with classification labels indicating whether the corresponding pixels depict a diamond, a different part of the diamond, a diamond holder, a background, a defect, a defect type, etc.

[0066] Through a data processing system (e.g., a computer similar to the analysis server 110a, data source 120, or end-user device 140 described in FIG. 1), the user may input the image sequence 502 into a machine learning model (e.g., a feature extraction machine learning model) and train the machine learning model to predict the classification for each pixel of the image sequence 502. The machine learning model can process the image sequence 502 and output a predicted image sequence 506 having labels for each pixel indicating a prediction as to whether the pixel represents a diamond, a diamond holder, a defect, a defect type, a background, etc. The data processing system can determine the difference between the predicted image sequence 506 and the ground truth image sequence 504 and use backpropagation techniques with a loss function to train the machine learning model based on this difference. Thus, the user can facilitate the training of a machine learning model for predicting or extracting features from an image or an image sequence of a diamond on a diamond holder in a diamond synthesis apparatus.

[0067] In some embodiments, the user can label the pixels with defect labels to indicate where the defects on the diamond are located and / or the type of such defects. For example, a machine learning model may be trained to predict the classification of pixels including diamond clean, diamond holder, diamond defect, diamond defect type, and background using a method similar to the above. The machine learning model can be trained to predict any classification. Through such training, the machine learning model may be able to predict whether there are defects in the diamond in the predicted image during post-processing, the type of such defects, and the location of the diamond defects.

[0068] It should be noted that the features extracted from the image sequence 502 may be a numerical vector output generated by a non-linear activation function (e.g., logistic). The extracted features have no visual meaning and may be aimed at helping the operator develop an optimization algorithm to enhance the convergence to the loss function used in the training of the machine learning model.

[0069] Here, referring to FIG. 6, a graph 600 is shown that displays the learning progress of a model trained to extract image features from an image. The graph 600 includes accuracy data of a machine learning model trained to classify the pixels of an image into a background class 602, a pocket holder class 604, and a diamond class 606 over the number of iterations (e.g., 50,000 times) of an image or a sequence of images. As shown, as the machine learning model receives and is trained based on additional images, the accuracy of the machine learning model improves. After training for a number of iterations, the machine learning model is shown to have a final average pixel discrimination accuracy of 99.8% for the background class 602, 99.44% for the pocket holder class 604, and 99.66% for the diamond class 606. Line 608 shows the total accuracy of the machine learning model for the training instances, indicating that the final average pixel discrimination accuracy is 99.7%.

[0070] Referring now to FIG. 7, a sequence 700 of machine learning models that predict a predicted image of a diamond based on a time series image of a diamond being synthesized within a diamond synthesis apparatus is illustrated. A sequence of images 702a - 702e may be input into a machine learning model (e.g., a network architecture including units for a long short - term memory network configured to make predictions about the future state of a diamond (e.g., future time steps) during synthesis and then make further predictions about the state of the diamond based on previous predictions). The sequence of images 702a - 702e may include images captured by an image capture device (e.g., a high - definition camera) configured to capture images of the diamond within the diamond synthesis apparatus at a preset interval (e.g., every 2 minutes). The sequence of images 702a - 702e may be input into the machine learning model, and the machine learning model may be able to output a predicted image 704 of the diamond (and other diamond crystallizations that may grow during synthesis) at a future time (e.g., 6 hours).

[0071] The actual image 706 shows the expected output of the machine learning model. As illustrated, the predicted image 704 is similar to the actual image 706, and thus, by implementing the systems and methods described herein, a data processing system can accurately predict how diamonds and other crystallizations will grow during synthesis using the machine learning model.

[0072] Referring now to FIG. 8, there is shown a diagram illustrating an image prediction model 800. The image prediction model 800 may be the same as or similar to the diamond state prediction model 160. The image prediction model 800 includes an encoder 802 and a decoder 804, and may be configured to generate a predicted image or another data object that indicates a predicted state of a diamond being synthesized after a predetermined number of time steps. The encoder 802 may be configured to integrate time series information from an input image sequence and a numerical sequence of reactor parameters into a fixed 14,400-dimensional vector. The decoder 804 may be configured to reconstruct an output RGB image corresponding to a predicted reactor state that is expected after a predetermined number of time steps into the future. The image prediction model 800 can be used to predict the future growth state 6 hours or any other time later, and in some cases can also be used with a verification accuracy of 99.99989%. For each arrow between the respective layers of the image prediction model 800, the text above each arrow can indicate the shape / dimension of the output data emerging from the previous layer, and the text below can mean the total number of trainable parameters used within the previous layer. The image prediction model 800 may have a total of 561 million trainable parameters. Thus, the image prediction model 800 can solve sequential time series problems with an exponentially increasing parameter space using diamond growth state prediction. This is because each new growth state of the diamond contributes to and potentially prioritizes the next growth state, yet still depends on the previous growth state of the diamond.

[0073] The parameters of a reactor (e.g., a diamond synthesis apparatus) are recorded at intervals of seconds and can include the input process gas, pressure, reactor temperature, input and reflected power, pocket depth and dimensions, lateral substrate dimensions, etc. An optical image recording device or an image capture device can capture images of the diamond being synthesized in the diamond synthesis apparatus at regular time intervals until the final stage of growth is executed. The image prediction model 800 may include a sequence-based growth prediction pipeline that predicts the state of the diamond being synthesized up to a future predetermined period (e.g., 1 hour, 2 hours, 3 hours, 6 hours, 12 hours, 1 day, 1 week, or any other arbitrary predetermined period). In one example, the system implementing the systems and methods described herein can achieve an accuracy of 99.99989% based on the highest average pixel loss achieved in the case of 6 hours. The system can make predictions using the following user-defined parameters: 1) Input data: A group of x image sequences at y-minute intervals; a numerical sequence of (5 + t - 1) temperature and pressure tuples at y-minute intervals. 2) Output data: A single output image frame (or other data object) of the predicted value of the model for the reaction state at the t-hour step ahead. Here, x and y are random variables determined after repeated training to determine the optimal values, and t is the number of time steps into the future for predicting the state of the diamond being synthesized.

[0074] Based on the input parameters, the model may be configured to obtain values for approximately 561 million trainable parameters, and as shown in FIG. 8, may employ an encoder-decoder architecture at a broad level. The encoder 802 may be optimized to integrate temporal information from the input image sequence 806 via layers 808, 810, 812, and 814 and a numerical sequence 816 (e.g., reactor parameters) via layers 818, 820, 822. The encoder 802 may use a concatenation layer 824 to concatenate the concatenated image sequence 806 and the numerical sequence 816 into a fixed multi-dimensional (e.g., 14,400- or 39,000-dimensional) vector 826. The decoder 804 may obtain the multi-dimensional vector 826 and pass the multi-dimensional vector 826 to an output layer 850 via layers 828, 830, 832, 834, 836, 838, 840, 842, 844, 846, and 848 of the decoder 804. The data processing system may obtain, at the output layer 850, a predicted image 852 (or another data object such as a multi-dimensional vector, value, etc.) indicating the predicted state of the diamond during synthesis at a future predetermined time.

[0075] When the image sequence 806 and the numerical sequence 816 are used for training, the data processing system may determine a training loss 854 of the predicted image (or data object) based on the difference between the predicted image and the expected image. The data processing system may use backpropagation techniques to train the image prediction model 800 based on the determined difference to improve the accuracy of the image prediction model 800. Thus, the image prediction model 800 can improve its accuracy over time while considering changes in how the diamond synthesis apparatus functions as the apparatus operates.

[0076] In summary, a network architecture consisting of units for long short-term memory networks (LSTMs) can encode a numerical environmental sequence into a 4,200-dimensional vector. For encoding an input image sequence, in a special implementation of the LSTM, dense units can be replaced with convolutional layers to learn spatial characteristics (convolutional LSTM). A series of convolutional LSTMs can encode information from the image sequence into a 34,800-dimensional vector (from the past state of diamond growth to a continuous mathematical entity) and combine it with the numerical sequence vector to obtain the input feature vector of the decoder network. This network can use a series of transposed convolutional 2D layers towards the future to upsample the low-dimensional encoder output into an output representing the state of the diamond after a predefined number of time steps. The entire network can be trained to minimize the mean squared loss between the pixels of the predicted image and the actual output image. Basically, the encoder and decoder can be designed to identify characteristic spatial features and sequentially transform them mathematically towards the future state based on the vector information obtained from the past sequence.

[0077] By controlling the feature quantities output from the encoder and optimizing the loss function to give high weights to the contours of the diamond and the pocket holder, the information loss between the predicted output and the actual output can be minimized, and a prediction accuracy of 99.99989% can be obtained. This is an unprecedented achievement of numerically predicting the shape of a diamond from the data obtained during growth rather than through simulation, demonstrating in-situ growth state prediction based on input data from the data collected during previous growth states.

[0078] In one embodiment, the method includes separating features of an optical image captured within a reactor (e.g., a diamond synthesis apparatus) using a deep learning (AI)-based segmentation or feature extraction model; using a deep learning (AI)-based frame prediction model with the image and high-dimensional and heterogeneous past state parameters including long parameters of the reactor to (1) predict a future growth state and (2) analyze a temporal separation evolution of features extracted from the growth state; and simulating an expression learned from the image segmentation or feature extraction and frame prediction models to find parameters for defect-free optimal diamond crystal growth.

[0079] In another embodiment, a software system can comprise a computer-implemented deep learning or AI-based algorithm that is used to separate and analyze features from an optical image captured within a reactor, predict a future growth state from reactor logs and images captured during diamond growth, analyze a temporal separation evolution of features extracted from the growth state, and simulate an expression learned from the model to detect parameters for defect-free optimal diamond crystal growth. The software-implemented architecture can include a streaming function that can be used for synchronization of data between the crystal synthesis reactor and a development environment for training the AI model. The software-implemented architecture can include a database architecture that can handle unstructured data that can be used to store all datasets and model files. The computer can include one or more graphics processing units (GPUs) that can be used to train one or more of the AI-based algorithms or models. One or more combinations of a CPU and a GPU can be used for testing / evaluating the trained models. A dashboard may be provided on the user interface for the manufacturing scientist to peruse. The dashboard can be developed while analyzing the results from the AI model and updated iteratively.

[0080] Overall, the systems and methods described herein provide an AI / deep learning-based pipeline consisting of algorithms for understanding, predicting, and optimizing diamond crystal growth. One aspect of the pipeline is an algorithm that extracts features of optical images captured within a reactor using image segmentation and a feature extraction model based on deep learning (AI). Another aspect of the pipeline is an algorithm using a deep learning (AI)-based frame prediction model with high-dimensional and heterogeneous past state parameters, which predicts future growth states and uses the predicted growth states to analyze the temporal evolution of features extracted from the growth states using the feature extraction model, including images and reactor long parameters. Another aspect of the pipeline is that it can find parameters for defect-free optimal diamond crystal growth by simulating the representations learned from image segmentation and feature extraction and the frame prediction model.

[0081] The pipeline provided may be a coupling unit within a software system modeled to develop and integrate the above deep learning or AI-based algorithms. One aspect of the software system is a software implementation architecture with a streaming function that can be used for data synchronization between a crystal synthesis furnace and a development environment for training an AI model. Another aspect of the software system is that a software implementation database architecture capable of handling unstructured data is used to store all data sets and model files. Another aspect of the software system includes one or more GPUs used to train an AI model, and a computing system where either the CPU or GPU or a combination thereof is used for testing / evaluating the trained AI model. In yet another aspect, a dashboard for manufacturing scientists to peruse and iteratively update the analysis regarding the results from the AI model is included.

[0082] The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed in this specification can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure or the claims.

[0083] Embodiments implemented in computer software can be implemented in software, firmware, middleware, microcode, hardware description language, or any combination thereof. Code segments or device-executable instructions can represent a procedure, function, subprogram, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. A code segment can be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. can be passed, transferred, or transmitted via any suitable means, such as memory sharing, message passing, token passing, network transmission, etc.

[0084] The actual software code or specialized control hardware used to implement these systems and methods does not limit the features described in the claims or the present disclosure. Accordingly, although the operation and behavior of the systems and methods have been described without reference to specific software code, it is understood that the software and control hardware can be designed to implement the systems and methods based on the description herein.

[0085] When implemented in software, a function can be stored as one or more instructions or codes on a non-transitory computer-readable or processor-readable storage medium. The steps of the methods or algorithms disclosed herein can be embodied in a processor-executable software module that can exist on a computer-readable or processor-readable storage medium. A non-transitory computer-readable medium or processor-readable medium includes both a computer storage medium and a tangible storage medium that facilitate the transfer of a computer program from one place to another. A non-transitory processor-readable storage medium can be any available medium that can be accessed by a computer. By way of example and not limitation, such non-transitory processor-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer or a processor. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disk typically reproduces data magnetically, while disc reproduces data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media. Further, the operations of a method or algorithm can exist as one or any combination or set of codes and / or instructions on a non-transitory processor-readable medium and / or a computer-readable medium and can be incorporated into a computer program product.

[0086] The foregoing description of the disclosed embodiments is provided to enable a person skilled in the art to make or use the embodiments described herein and their variations. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the principles defined herein can be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Accordingly, while the present disclosure is not intended to be limited to the embodiments shown herein, the broadest scope consistent with the following claims and the principles and novel features disclosed herein should be given.

[0087] Although various aspects and embodiments are disclosed, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed are for purposes of illustration and not intended to be limiting, and the true scope and spirit are indicated by the following claims.

Claims

Claim 1 A method for synthesizing diamond using a diamond synthesis apparatus, comprising: receiving, by a processor, a plurality of images of diamond being synthesized in the diamond synthesis apparatus, each of the plurality of images being captured within a period; executing, by the processor, a diamond state prediction machine learning model using the plurality of images to obtain a prediction data object, the prediction data object indicating a predicted state of the diamond in the diamond synthesis apparatus at a time after the period; detecting, by the processor, a predicted defect of the diamond based on the predicted state of the diamond; adjusting, by the processor, an operation of the diamond synthesis apparatus in response to the detection of the defect. Claim 2 The method according to claim 1, wherein the prediction data object comprises a prediction image depicting the diamond in the predicted state, the prediction image comprising a plurality of pixels, further comprising: executing, by the processor, a feature extraction machine learning model to extract a plurality of classification labels for the plurality of pixels of the prediction image; detecting the predicted defect in the predicted state of the diamond comprising detecting the predicted defect based on the plurality of classification labels. Claim 3 The method according to claim 2, wherein executing the feature extraction machine learning model to extract the plurality of classification labels comprises executing a defect detection machine learning model to extract a diamond top label, a diamond side label, a diamond holder label, a macroscopic defect label, a microscopic defect label, a center defect label, or an edge defect label for the plurality of pixels of the prediction image. Claim 4 The method according to claim 1, wherein executing the diamond state prediction machine learning model comprises executing, by the processor, the diamond state prediction machine learning model using the plurality of images and a schedule of operation parameters of the diamond synthesis apparatus to obtain the prediction data object, the schedule of operation parameters of the diamond synthesis apparatus corresponding to the time when the plurality of images are captured. Claim 5 The schedule of the operation parameters further includes a first set of operation parameters for the diamond synthesis apparatus for a first time between the period and the time following the period, the method according to claim 4.

6. The execution of the diamond state prediction machine learning model by the processor causing the diamond state prediction machine learning model to convert the plurality of images into a multi-dimensional vector corresponding to a predicted first state of the diamond at a first time step after the period and generating the predicted data object based on the multi-dimensional vector, the method according to claim 1.

7. The predicted data object includes a predicted image depicting the diamond in the predicted state, and detecting the predicted defect includes the processor evaluating the shape of the diamond depicted in the predicted image according to a diamond defect criterion, and in response to determining that the shape of the diamond meets the diamond defect criterion, the processor detecting a macroscopic defect, a microscopic defect, a central defect, or an edge defect of the diamond, the method according to claim 1.

8. Adjusting the operation of the diamond synthesis apparatus includes the processor adjusting the pressure or temperature of the diamond synthesis apparatus, the method according to claim 1.

9. The predicted data object includes a predicted image depicting the diamond in the predicted state, and the step of detecting the predicted defect in the diamond depicted in the predicted image includes the processor executing a feature extraction machine learning model using the predicted image to obtain a contour of the predicted defect in the diamond, the processor identifying the contour of the predicted defect, and the processor detecting the predicted defect in response to the identification of the contour, the method according to claim 1.

10. The predicted data object includes a predicted image depicting the diamond in the predicted state, and the step of detecting the predicted defect in the diamond depicted in the predicted image includes: executing, by the processor, a feature extraction machine learning model to obtain a contour of the diamond or a diamond holder using the predicted image; identifying, by the processor, a size or shape of the diamond based on the contour; and detecting, by the processor, the predicted defect in response to the identification of the size or shape. The method according to claim 1.

11. The method according to claim 1, wherein the predicted data object includes a value, a vector, or an image.

12. A system for synthesizing a diamond using a diamond synthesis apparatus, a processor configured to execute instructions stored on a non-transitory computer-readable medium, the processor receiving a plurality of images of a diamond during synthesis within the diamond synthesis apparatus, each of the plurality of images being captured within a period, using the plurality of images to execute a diamond state prediction machine learning model to obtain a predicted data object, the predicted data object indicating a predicted state of the diamond within the diamond synthesis apparatus at a time after the period, detecting a predicted defect within the diamond based on the predicted state of the diamond, and configured to adjust the operation of the diamond synthesis apparatus in response to the detection of the defect. A system comprising a processor.

13. The predicted data object includes a predicted image depicting the diamond in the predicted state, the predicted image including a plurality of pixels, the processor further configured to execute a defect detection machine learning model to extract a plurality of classification labels for the plurality of pixels of the predicted image, and the processor configured to detect the predicted defect within the diamond based on the plurality of classification labels for the plurality of pixels. The system according to claim 12.

14. The system according to claim 13, wherein the processor is configured to execute a feature extraction machine learning model to extract the plurality of classification labels by extracting, for the plurality of pixels of the prediction image, a diamond top label, a diamond side label, a diamond holder label, a macroscopic defect label, a microscopic defect label, a central defect label, or an edge defect label.

15. The system according to claim 12, wherein the processor is configured to execute a diamond state prediction machine learning model by using a schedule of the plurality of images and operation parameters of the diamond synthesis apparatus to obtain the prediction data object, and the schedule of the operation parameters of the diamond synthesis apparatus corresponds to a time when the plurality of images are captured.

16. The system according to claim 15, wherein the schedule of the operation parameters further includes a first set of operation parameters of the diamond synthesis apparatus for a first time between the period and the time following the period.

Images