An embryo image prediction method and device, an electronic device, and a storage medium

Abstract

The application provides an embryo image prediction method and device, electronic equipment and storage medium, which comprises: acquiring a to-be-processed embryo image, and extracting feature structured data from the to-be-processed embryo image; obtaining a kinetic parameter feature in the feature structured data using a kinetic parameter index, the kinetic parameter index being obtained by screening using a feature screening model; and predicting the kinetic parameter feature using a machine learning model to obtain a prediction result of the to-be-processed embryo image, the prediction result representing an embryo pregnancy probability corresponding to the to-be-processed embryo image. Since the kinetic parameter index is obtained by screening using the feature screening model, the feature dimension used in the evaluation and prediction of the embryo is effectively improved to be less subjective and less dimensional, thereby effectively improving the prediction accuracy of the embryo pregnancy corresponding to the embryo image.

Description

Technical Field

[0001] This application relates to the technical field of data mining and embryo image analysis, and more specifically, to an embryo image prediction method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, most embryo image classification methods involve manually examining the images and making assessments and predictions to determine whether the embryo has migrated to another environment (e.g., an isolation environment or inside a recipient). However, in practice, it has been found that due to limitations in human experience, the feature dimensions used in assessing and predicting embryos are relatively subjective and limited, making it difficult to improve the accuracy of predicting embryonic pregnancy corresponding to the images. Summary of the Invention

[0003] The purpose of this application is to provide an embryo image prediction method, apparatus, electronic device, and storage medium to improve the problem of difficulty in improving the prediction accuracy of embryonic pregnancy corresponding to embryo images.

[0004] This application provides an embryo image prediction method, comprising: acquiring an embryo image to be processed and extracting feature structured data from the embryo image; obtaining kinetic parameter features from the feature structured data using kinetic parameter indices, wherein the kinetic parameter indices are obtained by filtering using a feature selection model; and predicting the kinetic parameter features using a machine learning model to obtain a prediction result for the embryo image to be processed, wherein the prediction result represents the probability of pregnancy of the embryo corresponding to the embryo image to be processed. In the implementation of the above scheme, by using the kinetic parameter indices selected by the feature selection model to perform feature filtering and prediction on the feature structured data extracted from the embryo image to be processed, since the kinetic parameter indices are obtained by filtering using the feature selection model, the situation where the feature dimensions used in evaluating and predicting embryos are relatively subjective and have few dimensions is effectively improved. Therefore, using the kinetic parameter indices selected by the feature selection model for feature filtering and prediction can effectively improve the prediction accuracy of embryo pregnancy corresponding to the embryo image.

[0005] Optionally, in this embodiment, the feature selection model is an Extreme Gradient Boosting (XGBoost) model. Before using the kinetic parameter index to obtain the kinetic parameter features in the structured feature data, the method further includes: obtaining multiple weight parameters in the trained XGBoost model; for each weight parameter, determining whether the weight parameter is greater than a preset threshold; if so, determining the kinetic parameter field corresponding to the weight parameter as a kinetic parameter index. In the implementation of the above scheme, by determining the kinetic parameter index based on whether the weight parameters in the XGBoost model are greater than a preset threshold, duplicate feature indices in multiple kinetic parameter fields are removed while retaining the feature indices of the embryonic kinetic parameter state as much as possible, thereby reducing the training error of the feature index itself between different machine learning models.

[0006] Optionally, in this embodiment, before obtaining multiple weight parameters in the trained XGBoost model, the method further includes: obtaining kinetic parameter features of multiple sample embryo images and multiple sample pregnancy labels, where the sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image; using the kinetic parameter features of multiple sample embryo images as training data and the multiple sample pregnancy labels as training labels, determining the accuracy of multiple embryo prediction models, where the machine learning model is one of the multiple embryo prediction models; and adjusting the weight parameters of the XGBoost model with the accuracy of the multiple embryo prediction models as the objective function until the average accuracy of the multiple embryo prediction models rises to a preset stable range, thereby obtaining the XGBoost model, where each weight parameter of the XGBoost model corresponds to a kinetic parameter field. In the implementation of the above scheme, by training the XGBoost network with the accuracy of multiple embryo prediction models as the objective function and using the kinetic parameter indicators selected by the XGBoost network for feature selection and prediction, the prediction accuracy of embryo pregnancy corresponding to the embryo image can be effectively improved.

[0007] Optionally, in this embodiment, before using the machine learning model to predict the kinetic parameter features, the method further includes: acquiring multiple sample embryo images and multiple sample pregnancy labels, where the sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image; extracting a set of kinetic parameter features from each sample embryo image in the multiple sample embryo images to obtain multiple sets of kinetic parameter features; using kinetic parameter indicators to filter each set of kinetic parameter features in the multiple sets of kinetic parameter features to obtain multiple sets of filtered kinetic parameter features; and training the machine learning model using the multiple sets of filtered kinetic parameter features as training data and the multiple sample pregnancy labels as training labels. In the implementation of the above scheme, by using multiple sets of filtered kinetic parameter features as training data and multiple sample pregnancy labels as training labels to train the machine learning model, and using the machine learning model to predict the filtered kinetic parameter features, the prediction result of the embryo image to be processed is obtained, thereby improving the prediction accuracy of the embryo pregnancy corresponding to the embryo image.

[0008] Optionally, in this embodiment, training the machine learning model includes: constructing time series data based on the selected kinetic parameter features; performing fitting, completion, and consistency integration on the time series data to obtain consistent sequence data; and training the machine learning model using the consistent sequence data as training data and the sample pregnancy label as the training label. In the implementation of the above scheme, by constructing time series data based on the selected kinetic parameter features, the time series data can characterize the potential trends of kinetic parameter features, thus effectively balancing the interpretability of kinetic parameters and the accuracy of the machine learning model.

[0009] Optionally, in this embodiment, fitting and completing the time series data and performing consistency integration to obtain consistent series data includes: decomposing the time series data using a time series decomposition algorithm to obtain trend series data, periodic series data, and residual series data; fitting and completing the trend series data and periodic series data using a differentially integrated moving average autoregressive (ARIMA) model to obtain first completed data; fitting and completing the residual series data using a long short-term memory (LSTM) model to obtain second completed data; and performing consistency integration on the first and second completed data to obtain consistent series data. In the implementation of the above scheme, by using a differentially integrated moving average autoregressive (ARIMA) model to decompose the time series data and using a long short-term memory (LSTM) model to fit and complete the residual series data, the impact of time differences caused by the inconsistency between the recorded thawing time in the frozen embryo and the actual thawing time of the frozen embryo is minimized.

[0010] Optionally, in the embodiments of this application, the machine learning model includes: Extreme Gradient Boosting (XGBoost) model, Decision Tree ID3 model, Optimal Classification Tree (OCT) model, or Random Forest (RF) model.

[0011] This application also provides an embryo image prediction device, comprising: a structural data extraction module for acquiring an embryo image to be processed and extracting feature structured data from the embryo image to be processed; a parameter feature acquisition module for acquiring kinetic parameter features in the feature structured data using kinetic parameter indices, wherein the kinetic parameter indices are obtained by screening using a feature screening model; and an embryo image prediction module for predicting the kinetic parameter features using a machine learning model to obtain a prediction result of the embryo image to be processed, wherein the prediction result characterizes the embryo pregnancy probability corresponding to the embryo image to be processed.

[0012] Optionally, in this embodiment, the feature selection model is an Extreme Gradient Boosting (XGBoost) model; the embryo image prediction device further includes: a weight parameter acquisition module, used to acquire multiple weight parameters in the trained XGBoost model; a weight parameter judgment module, used to determine whether each weight parameter is greater than a preset threshold; and a parameter index determination module, used to determine the dynamic parameter field corresponding to the weight parameter as a dynamic parameter index if the weight parameter is greater than the preset threshold.

[0013] Optionally, in this embodiment, the embryo image prediction device further includes: an image label acquisition module, used to acquire the dynamic parameter features of multiple sample embryo images and multiple sample pregnancy labels, wherein the sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image; a network model training module, used to determine the accuracy of multiple embryo prediction models using the dynamic sample features as training data and the multiple sample pregnancy labels as training labels, wherein the machine learning model is one of the multiple embryo prediction models; and a weight parameter adjustment module, used to adjust the weight parameters of the XGBoost model with the accuracy of the multiple embryo prediction models as the objective function until the average accuracy of the multiple embryo prediction models rises to a preset stable range, thereby obtaining the XGBoost model, wherein each weight parameter of the XGBoost model corresponds to a dynamic parameter field.

[0014] Optionally, in this embodiment, the embryo image prediction device further includes: a training data acquisition module, used to acquire multiple sample embryo images and multiple sample pregnancy labels, wherein the sample pregnancy label is a category label of the embryo pregnancy result corresponding to the sample embryo image; a parameter feature extraction module, used to extract a set of dynamic parameter features from each sample embryo image in the multiple sample embryo images to obtain multiple sets of dynamic parameter features; a parameter feature filtering module, used to use dynamic parameter indicators to filter each set of dynamic parameter features in the multiple sets of dynamic parameter features to obtain multiple sets of filtered dynamic parameter features; and a model learning and training module, used to train a machine learning model using the multiple sets of filtered dynamic parameter features as training data and the multiple sample pregnancy labels as training labels.

[0015] Optionally, in this embodiment, the model learning and training module includes: a time series construction submodule, used to construct time series data based on the selected dynamic parameter features; a sequence data acquisition submodule, used to perform fitting, completion, and consistency integration on the time series data to obtain consistent sequence data; and a machine learning training submodule, used to train the machine learning model using the consistent sequence data as training data and the sample pregnancy label as training label.

[0016] Optionally, in this embodiment, the sequence data acquisition submodule includes: a first fitting and completion unit, used to decompose the time series data using a time series decomposition algorithm to obtain trend sequence data, periodic sequence data, and residual sequence data, and to fit and complete the trend sequence data and periodic sequence data using a differentially integrated moving average autoregressive ARIMA model to obtain first completed data; a second fitting and completion unit, used to fit and complete the residual sequence data using a long short-term memory LSTM model to obtain second completed data; and a completed data integration unit, used to perform consistency integration of the first completed data and the second completed data to obtain consistent sequence data.

[0017] Optionally, in the embodiments of this application, the machine learning model includes: Extreme Gradient Boosting (XGBoost) model, Decision Tree ID3 model, Optimal Classification Tree (OCT) model, or Random Forest (RF) model.

[0018] This application also provides an electronic device, including a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform the method described above.

[0019] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to perform the methods described above.

[0020] Other features and advantages of embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The flowchart shown is a schematic diagram of the embryo image prediction method provided in an embodiment of this application;

[0023] Figure 2 The diagram shown is a schematic diagram of the dynamic parameter fields before screening provided in an embodiment of this application;

[0024] Figure 3 A schematic diagram showing the filtered dynamic parameter indices provided in the embodiments of this application is shown;

[0025] Figure 4 The diagram shown illustrates the area under the ROC curve of the XGBoost model provided in this application embodiment under 10-fold cross-validation.

[0026] Figure 5 The diagram shown is a structural schematic of the embryo image prediction device provided in an embodiment of this application;

[0027] Figure 6 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in the embodiments of this application are for illustrative and descriptive purposes only and are not intended to limit the protection scope of the embodiments of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the embodiments of this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of the embodiments of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0029] Furthermore, the described embodiments are merely a part of the embodiments of this application, and not all of them. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of this application, but merely to illustrate selected embodiments of this application.

[0030] It is understood that the terms "first" and "second" in the embodiments of this application are used to distinguish similar objects. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different. In the description of the embodiments of this application, the term "and / or" is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. The term "multiple" refers to two or more (including two), and similarly, "multiple groups" refers to two or more groups (including two groups).

[0031] Before introducing the embryo image prediction method provided in the embodiments of this application, let's first introduce some concepts involved in the embodiments of this application:

[0032] A decision tree is a flowchart-like tree structure where each intermediate node represents a test on an attribute or a set of attributes, each branch corresponds to a different result of that test, and each leaf node represents a category or prediction result. Algorithms that generate decision trees from training data are usually called decision tree learning algorithms or decision tree algorithms.

[0033] Random forest refers to a classifier that uses multiple trees to train and predict samples. In machine learning, a random forest is a classifier that contains multiple decision trees, and its output class is determined by the mode of the classes output by the individual trees.

[0034] It should be noted that the embryo image prediction method provided in this application embodiment can be executed by an electronic device. Here, electronic device refers to a device terminal or server with the function of executing computer programs. Device terminals include, for example, smartphones, personal computers, tablets, personal digital assistants, or mobile internet devices. Servers refer to devices that provide computing services through a network. Servers include, for example, x86 servers and non-x86 servers. Non-x86 servers include, for example, mainframes, minicomputers, and UNIX servers.

[0035] The following describes the applicable scenarios for this embryo image prediction method. These scenarios include, but are not limited to: using this method to replace manual viewing and evaluation of embryo images; and obtaining a prediction result characterizing the pregnancy probability of the embryo corresponding to the image after prediction using this method. In the field of assisted reproduction, doctors need to determine whether to transfer the embryo corresponding to the image to another environment (e.g., an isolation environment or inside the recipient) based on professional medical knowledge and other reference factors. Therefore, the prediction result of this embryo image prediction method is only an intermediate result, and this intermediate result is only one of the reference factors for doctors. Other reference factors include, but are not limited to, dynamic reference factors, embryo morphological reference factors, recipient health factors, etc.

[0036] Please see Figure 1 The illustrated flowchart shows an embryo image prediction method provided in an embodiment of this application. The main idea of ​​this embryo image prediction method is to use a feature selection model to screen dynamic parameter indicators, which can improve the situation where the feature dimensions used in evaluating and predicting embryos are relatively subjective and limited, thereby effectively improving the prediction accuracy of embryonic pregnancy corresponding to the embryo image. The implementation of the above embryo image prediction method may include:

[0037] Step S110: Obtain the image of the embryo to be processed and extract the feature structured data from the image of the embryo to be processed.

[0038] Embryo images to be processed refer to the embryo images that need to be processed. These embryo images can be time-lapse images or images acquired in real time.

[0039] The implementation methods for obtaining the embryo image to be processed in step S110 above are as follows: First, the target embryo is scanned using an ultrasound scanner, or the target embryo is photographed using a terminal device such as a camera, video recorder, or color camera to obtain an image of the embryo to be processed; then the terminal device sends the embryo image to be processed to an electronic device, and the electronic device receives the embryo image sent by the terminal device. The electronic device can store the embryo image to be processed in a file system, database, or mobile storage device. Second, a pre-stored embryo image to be processed is obtained, specifically, for example, from a file system, database, or mobile storage device. Third, an embryo image to be processed is obtained using software such as a browser from the Internet, or by using other applications to access the Internet to obtain the embryo image to be processed.

[0040] For example, in step S110 above, the implementation of extracting feature structured data from the embryo image to be processed is as follows: It is understood that since the embryo image to be processed is not easily recorded and analyzed structured information, a feature fusion single-shot multi-box detector (FSSD) network or a YOLO network can be used to perform target detection and feature extraction on the embryo image to be processed, thereby obtaining feature structured data that is easy to record and analyze.

[0041] Step S120: Use dynamic parameter indices to obtain dynamic parameter features in the feature structured data. The dynamic parameter indices are obtained by filtering using a feature screening model.

[0042] Kinetic parameter indicators refer to kinetic-related indicators selected from parameter fields during embryonic development using a feature screening model. Examples of kinetic parameter fields include: the total time for embryonic development to five cells (represented as t5), the time for three-cell to four-cell development (represented as s2), the time for two-cell to four-cell development (represented as cc2), and so on.

[0043] Step S130: Use a machine learning model to predict the dynamic parameter features and obtain the prediction results of the embryo image to be processed. The prediction results represent the pregnancy probability of the embryo corresponding to the embryo image to be processed.

[0044] Understandably, the aforementioned embryo image prediction method uses machine learning models to predict kinetic parameter features. It can interpret the prediction results of the embryo image based on the weighted proportions of the contributions of the selected kinetic parameter features to the prediction results. Therefore, compared to using end-to-end neural network models to directly predict embryo images, which have lower interpretability because they predict directly from the image rather than from quantifiable structural feature data, the aforementioned embryo image prediction method improves upon this by directly predicting from the image itself. Thus, the machine learning model used in this method offers better interpretability of the kinetic parameters.

[0045] In the above implementation process, the embryo image prediction method uses a model to screen parameter indicators in the feature selection stage. That is, by using the kinetic parameter indicators screened by the feature screening model, the feature structured data extracted from the embryo image to be processed is used for feature screening and prediction. Since the kinetic parameter indicators are obtained by screening using the feature screening model, it effectively improves the situation where the feature dimensions used in evaluating and predicting embryos are relatively subjective and have few dimensions. Therefore, using the kinetic parameter indicators screened by the feature screening model for feature screening and prediction can effectively improve the prediction accuracy of embryo pregnancy corresponding to the embryo image.

[0046] As an optional implementation of the above-mentioned embryo image prediction method, considering that there are many types of kinetic parameter indicators (there are already 24 common kinetic parameter indicators), the feature selection model mentioned above can adopt the extreme gradient boosting (XGBoost) tree model, that is, use the XGBoost model to perform feature selection on the kinetic parameter indicators; before using the kinetic parameter indicators in step S120, the kinetic parameter indicators can also be obtained using the XGBoost model. This implementation may include:

[0047] Step S121: Obtain multiple weight parameters in the trained XGBoost model.

[0048] Please see Figure 2 The diagram shows a schematic of the kinetic parameter fields before screening provided in the embodiments of this application; the kinetic parameter fields in the figure can include four categories: pronuclear related, cleavage time point related, period maintenance time related, and cleavage time required related; each category of kinetic parameter fields includes many parameter fields, which can be referred to in the figure for details.

[0049] For example, step S121 can be implemented as follows: The trained XGBoost model can be obtained from a file system or database. The dynamic parameter fields corresponding to each weight parameter in this XGBoost model are dynamic parameter fields determined through expert consensus. Specifically, multiple weight parameters in the trained XGBoost model can be obtained using an executable program compiled or interpreted in a preset programming language. The programming languages ​​that can be used include, for example, C, C++, Java, BASIC, JavaScript, LISP, Shell, Perl, Ruby, Python, and PHP, etc.

[0050] Step S122: For each of the multiple weight parameters, determine whether the weight parameter is greater than a preset threshold.

[0051] Step S123: If the weight parameter is greater than the preset threshold, then the dynamic parameter field corresponding to the weight parameter is determined as a dynamic parameter index.

[0052] Please see Figure 3 The diagram illustrates the filtered kinetic parameter indices provided in this application embodiment. The implementation of steps S122 to S123 is as follows: In practical application, the XGBoost model can filter kinetic parameter indices based on the relationship weights between the data features corresponding to the kinetic parameter fields and the pregnancy outcome of the embryo image (i.e., the weight parameters in the XGBoost model). Specific examples of the kinetic parameter fields include: tPNa, tPNf, PB2, t2_s, t2, t3_s, t3, t4_s, t4, t5_s, t5, t6_s, t6, t7_s, t7, t8_s, t8, t9, tSC, tM, t2_c, t3_c, t4_c, t5_c, t6_c, t7_c, t8_c, cycle2, cycle3, cycle4, cycle5, cycle6, cycle7, and cycle8, etc. The kinetic parameters selected and fitted above may include: tPNa, PB2, t2_s, t2, t3_s, t3, t4_s, t4, t5_s, t6_s, t2_c, t3_c, t4_c, t5_c, t6_c, t7_c, cycle2, cycle3, cycle4, and cycle5.

[0053] In the above implementation process, the dynamic parameter index is determined by whether the weight parameters in the XGBoost model are greater than a preset threshold. This removes duplicate feature indices from multiple dynamic parameter fields while retaining as many feature indices of embryo dynamic parameter state as possible, thereby reducing the training error of the feature indices between different machine learning models.

[0054] Optionally, in practical applications, decision tree models can be used to filter multiple dynamic parameter indicators to obtain dynamic parameter indicators. These decision tree models include, but are not limited to, ID3, Optimal Classification Tree (OCT), Random Forest (RF), Regression Decision Tree (RDT), or Gradient Boosting Decision Tree (GBDT), etc. In the above implementation process, the weights corresponding to the filtered dynamic parameter indicators can be obtained and used as reference indicators for optimizing the aforementioned machine learning model. These weights can also serve as the interpretive basis for end-to-end neural network models or deep learning models, thereby increasing the interpretability of the filtered dynamic parameter indicators.

[0055] As an optional implementation of the above-mentioned embryo image prediction method, the XGBoost model can be trained before using it to filter multiple dynamic parameter fields. This implementation may include:

[0056] Step S121a: Obtain the dynamic parameter features of multiple sample embryo images and multiple sample pregnancy labels. The sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image.

[0057] The aforementioned dynamic parameter fields include, for example: tPNa, tPNf, PB2, t2_s, t2, t3_s, t3, t4_s, t4, t5_s, t5, t6_s, t6, t7_s, t7, t8_s, t8, t9, tSC, tM, t2_c, t3_c, t4_c, t5_c, t6_c, t7_c, t8_c, cycle2, cycle3, cycle4, cycle5, cycle6, cycle7, and cycle8, etc.

[0058] Step S121b: Using the dynamic parameter features of multiple sample embryo images as training data and the pregnancy labels of multiple samples as training labels, determine the accuracy of multiple embryo prediction models. The machine learning model is one of the multiple embryo prediction models.

[0059] Understandably, during the training of the embryo prediction model, the goal is to maximize the average accuracy of multiple embryo prediction models. Then, when using these models for prediction, any one of them can achieve a high accuracy rate. Therefore, regardless of which embryo prediction model is used as the machine learning model to predict kinetic parameters, good prediction results can be achieved, meaning high prediction accuracy. Alternatively, the average accuracy of multiple embryo prediction models can also be maximized, ensuring that most of these models achieve good prediction results (i.e., high prediction accuracy) when used as machine learning models to predict kinetic parameters.

[0060] Step S122e: Using the accuracy of multiple embryo prediction models as the objective function, adjust the weight parameters of the XGBoost model until the average accuracy of multiple embryo prediction models rises to a preset stable range, and obtain the trained XGBoost model. Each weight parameter of the XGBoost model corresponds to a dynamic parameter field.

[0061] The implementation of steps S122d to S122e above is as follows: Multiple embryo prediction models are used to predict the embryonic pregnancy outcome corresponding to the kinetic parameter field of the sample embryo image, obtaining the predicted pregnancy outcome. Then, the average accuracy of the multiple embryo prediction models is calculated based on the predicted pregnancy outcome and the embryonic pregnancy outcome in the sample pregnancy label. The model parameters of the XGBoost model are updated based on the change in the average accuracy of the multiple embryo prediction models until the average accuracy of the multiple embryo prediction models rises to a preset stable range or the number of iterations (epochs) of the XGBoost model used for screening is greater than a preset threshold. At this point, the trained XGBoost model is obtained, where each weight parameter of the XGBoost model corresponds to a kinetic parameter field. The preset threshold can also be set according to specific circumstances, such as 100 or 1000, etc.

[0062] Understandably, if the average accuracy of the XGBoost network in this round is lower than the average accuracy of the previous training round, then steps S122b to S122e need to be re-executed. This involves re-filtering multiple kinetic parameter fields using the XGBoost model to obtain multiple re-filtered fields, and then using these re-filtered fields to obtain kinetic sample features from the structured sample data. Using these kinetic sample features as training data and multiple sample pregnancy labels as training labels, the accuracy of multiple embryo prediction models is determined. The machine learning model is one of these multiple embryo prediction models. Using the average accuracy of the multiple embryo prediction models as the objective function, the weight parameters of the XGBoost model are adjusted until the average accuracy of the multiple embryo prediction models rises to a preset stable range or the number of iterations (epochs) of the XGBoost model used for filtering exceeds a preset threshold. This yields the trained XGBoost model, where each weight parameter corresponds to a kinetic parameter field. The preset stable range can be set according to specific circumstances, for example, setting it to 70% to 90%, or 80% to 85%, etc. The resulting XGBoost model effectively identifies the weighting of kinetic parameters during embryonic development, with the highest-weighted parameters being those most influential on pregnancy outcomes. This process can be understood as identifying the kinetic parameters used by most high-accuracy machine learning models. The XGBoost model filters these parameters based on their weighting during embryonic development, effectively reducing training errors between different machine learning models.

[0063] Please see Figure 4 The diagram illustrates the area under the ROC curve of the XGBoost model provided in this embodiment under 10-fold cross-validation; where the ROC curve refers to the Receiver Operating Characteristic (ROC) curve. The XGBoost model can be trained on a training dataset with 10-fold cross-validation, in which case the area under the ROC curve (AUC) of the XGBoost model is relatively stable. As shown in the figure, the mean AUC of the XGBoost model is between 0.75 and 0.85, with a minimum of 0.70 and a maximum of 0.91. Therefore, the AUC of the XGBoost model is relatively stable under 10-fold cross-validation.

[0064] As an optional implementation of the above-described embryo image prediction method, before using the machine learning model for prediction in step S130, the machine learning model can be trained. This implementation may include:

[0065] Step S131: Obtain multiple sample embryo images and multiple sample pregnancy labels. The sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image.

[0066] For example, the above-mentioned step S131 can be implemented by obtaining multiple sample embryo images and multiple sample pregnancy tags from the file system and / or database. Multiple sample embryo images and multiple sample pregnancy tags can also be used as training datasets to train the machine learning model. During the training process, the hyperparameters of the machine learning model can be continuously adjusted and optimized.

[0067] Step S132: Extract a set of dynamic parameter features from each of the multiple sample embryo images to obtain multiple sets of dynamic parameter features.

[0068] An example of implementing step S132 above is as follows: It is understood that the sample embryo image can be an embryo image from a video. FSSD networks and YOLO networks can be used to perform target detection on the sample embryo image to obtain the current state and current time of the sample embryo. Then, the dynamic parameter features can be determined based on the video recording time, the current state of the sample embryo, and the current time. Therefore, by performing the above operation on each sample embryo image in the multiple sample embryo images, a set of dynamic parameter features can be extracted from each sample embryo image in the multiple sample embryo images, thus ultimately obtaining multiple sets of dynamic parameter features.

[0069] Step S133: Use dynamic parameter indices to filter each dynamic parameter feature in multiple sets of dynamic parameter features to obtain multiple sets of filtered dynamic parameter features.

[0070] It is understandable that the above-mentioned dynamic parameter indices are obtained by using a feature selection model, such as the XGBoost model. Therefore, the selected dynamic parameter indices can be directly used to perform feature selection on multiple sets of dynamic parameter features.

[0071] For example, the implementation of step S133 above is as follows: Assuming that there are 20 dynamic parameter indices, for each group of dynamic parameter features in the multiple groups of dynamic parameter features, the parameter feature data of these 20 dynamic parameter indices can be screened from the group of dynamic parameter features to obtain multiple groups of screened dynamic parameter features.

[0072] Step S134: Using multiple sets of screened dynamic parameter features as training data and multiple sample pregnancy labels as training labels, train the machine learning model to obtain the trained machine learning model.

[0073] The implementation of step S134 above is as follows: For each group of selected kinetic parameter features, a machine learning model is used to predict the selected kinetic parameter features of that group, obtaining the prediction result corresponding to that group of selected kinetic parameter features. Then, the loss value between the prediction result and the pregnancy result corresponding to the sample pregnancy label is calculated, and the network weight parameters of the machine learning model are updated according to the loss value until the average accuracy of multiple embryo prediction models rises to a stable range or the number of iterations (epochs) of the XGBoost model used for selection is greater than a preset threshold, at which point the trained machine learning model is obtained. The machine learning model mentioned above can be an Extreme Gradient Boosting (XGBoost) model, a Decision Tree ID3 model, an Optimal Classification Tree (OCT) model, a Random Forest (RF) model, a Regression Tree (RDT) model, or a GBDT model, etc. The preset threshold can also be set according to specific circumstances, such as 100 or 1000, etc.

[0074] As an optional implementation of step S134 above, the implementation of training the machine learning model may include:

[0075] Step S134a: Construct time series data based on the selected dynamic parameter characteristics.

[0076] Step S134b: Perform fitting, completion, and consistency integration on the time series data to obtain consistent series data.

[0077] An example implementation of step S134a above is as follows: An executable program compiled or interpreted using a preset programming language is used to construct time series data based on the selected dynamic parameter characteristics. The time series data is then fitted, completed, and integrated for consistency to obtain consistent sequence data. The programming languages ​​that can be used include, for example, C, C++, Java, BASIC, JavaScript, LISP, Shell, Perl, Ruby, Python, and PHP, etc.

[0078] Step S134c: Using consistent sequence data as training data and sample pregnancy labels as training labels, train the machine learning model to obtain the trained machine learning model.

[0079] An example implementation of step S134c above is as follows: A machine learning model is used to predict the consistent sequence data to obtain a predicted pregnancy result. The loss value between the predicted pregnancy result and the pregnancy result in the sample pregnancy label is calculated. The model weight parameters of the machine learning model are then updated based on this loss value until the average accuracy of multiple embryo prediction models rises to a preset stable range or the number of iterations (epochs) of the XGBoost model used for screening exceeds a preset threshold. The trained machine learning model is then obtained. The preset threshold can be set according to specific circumstances, such as 100 or 1000.

[0080] In the above implementation process, time series data is constructed based on the selected dynamic parameter features. Since time series data can characterize the potential dynamic parameter feature trends, it effectively balances the interpretability of dynamic parameters and the accuracy of machine learning models.

[0081] As an optional implementation of step S134b above, the implementation of fitting, completion, and consistency integration of time series data may include:

[0082] Step S134d: Use a time series decomposition algorithm to decompose the time series data to obtain trend series data, periodic series data and residual series data, and use the differential integrated moving average autoregressive ARIMA model to fit and complete the trend series data and periodic series data to obtain the first completed data.

[0083] The above step S134d can be implemented as follows: A time series decomposition algorithm is used to decompose the time series data to obtain trend series data, periodic series data, and residual series data. The time series decomposition algorithm used here can be a loss-based seasonal trend decomposition procedure (STL) algorithm. Then, a difference-integrated moving average autoregressive ARIMA model is used to fit and complete the trend series data and periodic series data, thereby completing the inconsistencies in the general direction of developmental stages in the data, ultimately obtaining the first complete data with consistent developmental stages.

[0084] Step S134e: Use a Long Short-Term Memory (LSTM) model to fit and complete the residual sequence data to obtain the second completed data.

[0085] It is understandable that the development stage of the second completed data is shorter than that of the first completed data. The two are fitted and completed using different models, therefore, the development stages of the two completed data are different.

[0086] For example, the above step S134e can be implemented by using a Long Short-Term Memory (LSTM) network model or a Bidirectional Long Short-Term Memory (Bi-LSTM) network model to fit and complete the residual sequence data, thereby completing the small directions of inconsistent developmental stages in the data, and finally obtaining the second completed data with consistent developmental stages, effectively mining the potential trend of the data and the hidden temporal feature trend.

[0087] Step S134f: Perform consistent integration on the first and second completed data to obtain consistent sequence data.

[0088] The implementation of step S134f above is as follows: It is understood that the LSTM network model or Bi-LSTM network model described above can uncover the potential trends and hidden temporal characteristics of the data. These trends can be input into the ARIMA model as time series data to form an ARIMA-LSTM time series model that completes each temporal feature. Then, the ARIMA-LSTM time series model is used to consistently integrate the first and second completed data to obtain a complete and low-noise consistent sequence data.

[0089] In the aforementioned process, due to variations in the timing of manual handling of a batch of frozen embryos, the recorded thawing time of the frozen embryo differs from its actual thawing time. This means that different embryos placed in the culture dish thaw at different times, leading to inconsistent developmental start times among different embryos from the same fertilized egg. However, in reality, embryo transfer is typically performed only once at a specific time. To mitigate the impact of these non-developmental time differences, a differential integrated moving average autoregressive (ARIMA) model is used to decompose the time series data, and a long short-term memory (LSTM) model is used to fit and complete the residual sequence data, thereby minimizing the impact of these non-developmental time differences.

[0090] Alternatively, as an optional implementation of the above-mentioned embryo image prediction method, the above-mentioned machine learning model may include, but is not limited to: Extreme Gradient Boosting (XGBoost) model, Decision Tree ID3 model, Optimal Classification Tree (OCT) model, or Random Forest (RF) model, etc.

[0091] Please see Figure 5 The diagram shown is a structural schematic of the embryo image prediction device provided in an embodiment of this application; this application provides an embryo image prediction device 200, including:

[0092] The structural data extraction module 210 is used to acquire the embryo image to be processed and extract the feature structured data from the embryo image to be processed.

[0093] The parameter feature acquisition module 220 is used to acquire dynamic parameter features in the feature structured data using dynamic parameter indices, which are obtained by filtering using a feature screening model.

[0094] The embryo image prediction module 230 is used to predict the dynamic parameter features using a machine learning model to obtain the prediction result of the embryo image to be processed. The prediction result represents the pregnancy probability of the embryo corresponding to the embryo image to be processed.

[0095] Optionally, in this embodiment, the feature selection model is an XGBoost model; the embryo image prediction device further includes:

[0096] The weight parameter acquisition module is used to obtain multiple weight parameters in the trained XGBoost model.

[0097] The weight parameter judgment module is used to determine whether each of the multiple weight parameters is greater than a preset threshold.

[0098] The parameter index determination module is used to determine the dynamic parameter field corresponding to the weight parameter as a dynamic parameter index if the weight parameter is greater than a preset threshold.

[0099] Optionally, in embodiments of this application, the embryo image prediction device further includes:

[0100] The image label acquisition module is used to acquire the dynamic parameter features of multiple sample embryo images and multiple sample pregnancy labels. The sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image.

[0101] The network model training module is used to determine the accuracy of multiple embryo prediction models by using the dynamic parameter features of multiple sample embryo images as training data and multiple sample pregnancy labels as training labels. The machine learning model is one of the multiple embryo prediction models.

[0102] The weight parameter adjustment module is used to adjust the weight parameters of the XGBoost model with the accuracy of multiple embryo prediction models as the objective function, until the average accuracy of multiple embryo prediction models rises to a preset stable range, thus obtaining the XGBoost model. Each weight parameter of the XGBoost model corresponds to a kinetic parameter field.

[0103] Optionally, in embodiments of this application, the embryo image prediction device further includes:

[0104] The training data acquisition module is used to acquire multiple sample embryo images and multiple sample pregnancy labels. The sample pregnancy label is the category label of the embryo pregnancy result corresponding to the sample embryo image.

[0105] The parameter feature extraction module is used to extract a set of dynamic parameter features from each of the multiple sample embryo images to obtain multiple sets of dynamic parameter features.

[0106] The parameter feature filtering module is used to filter each set of dynamic parameter features from multiple sets of dynamic parameter features using dynamic parameter indices, and obtain multiple sets of filtered dynamic parameter features.

[0107] The model learning and training module is used to train the machine learning model using multiple sets of screened dynamic parameter features as training data and multiple sample pregnancy labels as training labels.

[0108] Optionally, in this embodiment of the application, the model learning and training module includes:

[0109] The time series construction submodule is used to construct time series data based on the filtered dynamic parameter characteristics.

[0110] The sequence data acquisition submodule is used to fit, complete, and integrate time series data to obtain consistent sequence data.

[0111] The machine learning training submodule is used to train the machine learning model using consistent sequence data as training data and sample pregnancy labels as training labels.

[0112] Optionally, in this embodiment of the application, the sequence data acquisition submodule includes:

[0113] The first fitting and completion unit is used to decompose the time series data using a time series decomposition algorithm to obtain trend series data, periodic series data and residual series data, and to fit and complete the trend series data and periodic series data using a differential integrated moving average autoregressive ARIMA model to obtain the first completed data.

[0114] The second fitting and completion unit is used to fit and complete the residual sequence data using a long short-term memory (LSTM) model to obtain the second completed data.

[0115] The data completion integration unit is used to perform consistency integration of the first and second completed data to obtain consistent sequence data.

[0116] Optionally, in the embodiments of this application, the machine learning model includes: Extreme Gradient Boosting (XGBoost) model, Decision Tree ID3 model, Optimal Classification Tree (OCT) model, or Random Forest (RF) model.

[0117] It should be understood that this device corresponds to the above-described embryo image prediction method embodiment and is capable of performing the various steps involved in the above method embodiment. The specific functions of this device can be found in the description above, and detailed descriptions are appropriately omitted here. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0118] Please see Figure 6 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0119] This application embodiment also provides a computer-readable storage medium 330, on which a computer program is stored. This computer program is executed by a processor 310 to perform the methods described above. The computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0120] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0121] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, as provided in the embodiments of this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending primarily on the functions involved.

[0122] Furthermore, the functional modules of each embodiment in this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," "some examples," etc., means that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0123] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. An embryo image prediction method, characterized by, The method comprises the following steps: obtaining an image of an embryo to be processed, and extracting feature structured data from the image of the embryo to be processed; obtaining a kinetic parameter feature in the feature structured data using a kinetic parameter index, the kinetic parameter index being obtained from the feature structured data according to a plurality of kinetic parameter fields using a feature screening model; using a machine learning model to predict the kinetic parameter feature, and obtaining a prediction result of the image of the embryo to be processed, the prediction result representing a pregnancy probability of an embryo corresponding to the image of the embryo to be processed; wherein the feature screening model is an extreme gradient boosting XGBoost model, and the extraction of the feature structured data from the image of the embryo to be processed comprises feature extraction of the image of the embryo to be processed using a neural network to obtain the feature structured data.

2. The method of claim 1, wherein, The feature screening model is an extreme gradient boosting XGBoost model; before the step of obtaining the kinetic parameter feature in the feature structured data using the kinetic parameter index, the method further comprises the following steps: obtaining a plurality of weight parameters in the trained XGBoost model; for each weight parameter in the plurality of weight parameters, determining whether the weight parameter is greater than a preset threshold; if yes, determining a kinetic parameter field corresponding to the weight parameter as one of the kinetic parameter indexes.

3. The method of claim 2, wherein, Before the step of obtaining the plurality of weight parameters in the trained XGBoost model, the method further comprises the following steps: obtaining kinetic parameter features of a plurality of sample embryo images and a plurality of sample pregnancy labels, the sample pregnancy label being a category label of an embryo pregnancy result corresponding to the sample embryo image; determining an accuracy rate of a plurality of embryo prediction models by taking the kinetic parameter features of the plurality of sample embryo images as training data and taking the plurality of sample pregnancy labels as training labels, the machine learning model being one of the plurality of embryo prediction models; adjusting the weight parameters of the XGBoost model by taking the accuracy rates of the plurality of embryo prediction models as an objective function until the average accuracy rate of the plurality of embryo prediction models rises to a preset stable interval to obtain the XGBoost model, each weight parameter of the XGBoost model corresponding to a kinetic parameter field.

4. The method of claim 1, wherein, Before the step of using the machine learning model to predict the kinetic parameter feature, the method further comprises the following steps: obtaining a plurality of sample embryo images and a plurality of sample pregnancy labels, the sample pregnancy label being a category label of an embryo pregnancy result corresponding to the sample embryo image; extracting a group of kinetic parameter features from each sample embryo image in the plurality of sample embryo images to obtain a plurality of groups of kinetic parameter features; screening each group of kinetic parameter features in the plurality of groups of kinetic parameter features using the kinetic parameter index to obtain a plurality of groups of screened kinetic parameter features; training the machine learning model by taking the plurality of groups of screened kinetic parameter features as training data and taking the plurality of sample pregnancy labels as training labels.

5. The method of claim 4, wherein, The training of the machine learning model comprises the following steps: According to the screened kinetic parameter characteristics, time series data is constructed; The time series data is fitted, completed and consistency integrated to obtain consistent sequence data; The machine learning model is trained by taking the consistent sequence data as training data and the sample pregnancy label as training label.

6. The method of claim 5, wherein, The fitting, completion and consistency integration of the time series data to obtain consistent sequence data comprises: The time series data is decomposed by using a time series decomposition algorithm to obtain trend sequence data, periodic sequence data and residual sequence data, and the trend sequence data and the periodic sequence data are fitted and completed by using a difference integration moving average autoregressive ARIMA model to obtain first completed data; The residual sequence data is fitted and completed by using a long short-term memory LSTM model to obtain second completed data; The first completed data and the second completed data are consistency integrated to obtain the consistent sequence data.

7. The method according to any of claims 1 to 6, characterized in that The machine learning model comprises an extreme gradient boosting XGBoost model, a decision tree ID3 model, an optimal classification tree OCT model or a random forest RF model.

8. An embryo image prediction device, characterized by, It comprises: A structural data extraction module is configured to obtain a to-be-processed embryo image and extract feature structured data from the to-be-processed embryo image; A parameter feature acquisition module is configured to acquire kinetic parameter characteristics in the feature structured data by using kinetic parameter indexes, wherein the kinetic parameter indexes are obtained from the feature structured data by using a feature screening model according to multiple kinetic parameter fields; An embryo image prediction module is configured to predict the kinetic parameter characteristics by using a machine learning model to obtain a prediction result of the to-be-processed embryo image, wherein the prediction result represents a pregnancy probability of an embryo corresponding to the to-be-processed embryo image. The feature screening model is an extreme gradient boosting XGBoost model, and the feature structured data is extracted from the to-be-processed embryo image by using a neural network to extract features of the to-be-processed embryo image and obtain the feature structured data.

9. An electronic device, comprising: It comprises: A processor and a memory, wherein the memory stores machine readable instructions executable by the processor, and the machine readable instructions are executed by the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the method of any one of claims 1 to 7.

Images