Prediction method and medium of female fertility suitability based on sample cross attention

Abstract

The present invention provides a method and medium for predicting female fertility suitability based on sample cross-attention, comprising collecting a dataset of female physical examination medical records, dividing the dataset into a training set and a test set as training data, and performing data processing on the training data; constructing a data embedding model based on the word2vec algorithm and generating an embedding vector; constructing a feature extraction module based on inter-sample cross-attention to extract features from the embedding vector, and passing these features to a fertility suitability prediction module for prediction; training the above-mentioned fertility suitability prediction module using the collected training samples; inputting the data-processed medical record data into the trained fertility suitability prediction module to obtain a prediction result. The present invention not only solves the problem of insufficient analysis of inter-sample correlations faced by traditional deep learning models in structured data, but also achieves accurate prediction of fertility suitability, providing effective advice to women on whether to choose to have children.

Description

Technical Field

[0001] The present invention relates to the field of computer natural language processing, and in particular to a method, device and storage medium for predicting female reproductive suitability based on sample cross-attention. Background Art

[0002] Fertility suitability assessments are crucial for family planning and population health. Fertility is more than just a physiological process; it carries with it enormous responsibilities, particularly when it comes to raising children. Fertility suitability assessments help couples make more informed decisions, ensuring they have the resources, abilities, and psychological preparation to welcome a new life. Furthermore, fertility suitability assessments include screening for potential genetic diseases, which is crucial for protecting family health and preventing the spread of genetic diseases.

[0003] Traditional methods for assessing fertility suitability typically rely on a series of basic indicators, such as a woman's age, body mass index (BMI), medical history, pre-pregnancy physical examination data, and genetic screening results. However, these methods have limitations, primarily due to their inability to comprehensively consider multiple, complex factors and their over-reliance on physician experience and judgment. Traditional methods fail to fully leverage the potential of modern medical data and technology, resulting in the inability to provide accurate and personalized assessment results in some cases.

[0004] With the rapid development of deep learning technology, deep learning-based fertility suitability assessment methods have gradually gained attention. Deep learning can process more complex multidimensional data and capture nonlinear relationships between variables, thereby providing more accurate fertility suitability assessments. However, deep learning models are essentially "black box" models. While they can provide highly accurate predictions, their decision-making process lacks transparency, which may reduce doctors' and patients' trust in the model's recommendations. Furthermore, deep learning models typically require large amounts of high-quality data for training, and this data often involves sensitive personal health information. Effectively ensuring data privacy and security has become a major challenge in the application of this technology.

[0005] Therefore, the research and application of female fertility suitability prediction methods based on the cross-attention mechanism between samples has important practical significance. By combining deep learning with the cross-attention mechanism between samples, it can better mine multi-level information in the data, thereby improving the accuracy and reliability of predictions, while also enhancing the interpretability of the model and increasing the trust of doctors and patients. However, current technologies still face numerous challenges in terms of trust and privacy protection, and have not yet been widely adopted in clinical practice. Ensuring the accuracy of the technology while improving its transparency and privacy protection is a key issue that needs to be addressed. Summary of the Invention

[0006] The present invention proposes a method, device and storage medium for predicting female reproductive suitability based on sample cross-attention, which can solve at least one of the technical problems in the background technology; specifically, it can solve the problem that existing technologies do not fully utilize deep learning to predict the reproductive suitability of female patients.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for predicting female fertility suitability based on sample cross attention, performing the following steps by a computer device,

[0009] S1. Collect a dataset of female physical examination medical records, including medical record text data and labels, divide the dataset into a training set and a test set, and process the training data;

[0010] S2, build a data embedding model based on the word2vec algorithm and pass the generated embedding vector to subsequent structural processing;

[0011] S3. Construct a feature extraction module based on cross-attention between samples to extract features from the embedding vector and pass these features to the fertility suitability prediction module for prediction;

[0012] S4. Using the collected training samples to train the above prediction model;

[0013] S5. When applying, the medical record data after data processing is input into the prediction model to obtain the prediction results.

[0014] Furthermore, the step S1 includes:

[0015] S11. Collect medical records of women’s physical examinations, extract information related to health scores and reproductive health scores, and save them as a Word file;

[0016] S12. Extract the health score and reproductive health score from the Word file using the Python library python-docx, and save the health score and reproductive health score as an Excel file using the Python library openpyxl;

[0017] S13, dividing the data set into two parts: training data and test data;

[0018] S14. The medical record data is used as the model input, and the output of the fertility suitability prediction module is used as the final prediction result of the model.

[0019] Furthermore, the step S2 includes:

[0020] S21. Build a text embedding module based on the word2vec algorithm to convert text into embedding vectors;

[0021] S22. The text embedding module accepts tabular data as input. ,in To include A tabular dataset of samples, each is a The corresponding eigenvector of It’s a label;

[0022] S23. Add a classification tag with learned embedding to each data sample .make , is a data point with categorical features;

[0023] S24, the feature vector Input to an embedding layer In , each feature vector is embedded into d-dimensional space. The embedded feature is represented as ; where input , through the embedding layer Output after .

[0024] Furthermore, step S3 includes:

[0025] S31. Construct a feature extraction module based on cross-attention between samples and a fertility suitability prediction module. Feature representation after embedding based on the input of the feature extraction module based on cross-attention between samples , output the extracted features ; Input features of the fertility suitability prediction module , output the prediction result;

[0026] S32, the feature extraction module based on cross-sample attention consists of a multi-head self-attention layer, a multi-head cross-sample attention layer and two feedforward neural network layers;

[0027] S33, multi-head self-attention layer receives embedded feature representation As input, after multi-head self-attention processing and normalization, the residual connection outputs the result The formula is as follows:

[0028] ,

[0029] in, For the The results of these processing of samples are for Normalization processing, It is multi-head self-attention processing;

[0030] Multi-head self-attention processing : For each sample , each self-attention head has a trainable parameter matrix 、 and , respectively calculated 、 and Get the query matrix , key matrix Sum Matrix . First calculate the dot product , which represents the correlation between different features of the sample. In order to make the gradient of the training process more stable, it is scaled and then Normalization processing, each value becomes a weight coefficient greater than 0 and less than 1, that is, . Multiply the value matrix by the obtained weight coefficient matrix to get:

[0031] ,

[0032] in Indicates the value that needs to be corrected for different features of the sample. The results of multiple self-attention heads are concatenated, that is, . Then perform a linear transformation to restore it to its original dimension ,Right now , , , for subsequent calculations;

[0033] In multi-head self-attention processing Afterwards, the above results conduct Normalization and residual connection;

[0034] S34, the feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows:

[0035] ,

[0036] in, For the The result of samples processed by the multi-head self-attention layer, for Normalization processing, The feed-forward neural network (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation.

[0037] The following is the detailed process:

[0038] first, Input into the feedforward neural network, in the first layer, the feature dimension of the input Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to the original dimension ;

[0039] In feedforward neural network processing Afterwards, the above results were Normalization and residual connection;

[0040] S35, the multi-head cross-sample attention layer receives the results of the feedforward neural network layer As input, the obtained results are processed by multi-head cross-sample attention, normalized, and finally the residual connection output is performed. The formula is as follows:

[0041] ,

[0042] in, For the The result of samples processed by the feedforward neural network layer, for Normalization processing, It is a multi-head cross-sample attention process. Unlike self-attention, cross-sample attention is calculated for different data points (i.e. different samples) in a given batch, not just the features of a single data point. Specifically, we concatenate the embedding vector of each feature to a single data point and then calculate the attention value of the sample. This allows us to improve the representation of a given point by examining other points. When some features of a sample are missing or noisy, cross-sample attention can borrow corresponding features from other similar samples in the batch;

[0043] Multi-head cross-sample attention processing :for , randomly select a sample , calculate the cross attention value between the sample and all samples, and then perform Normalization processing, that is ,in express The query matrix, Represents all value matrices. Multiplying the obtained weight coefficient matrix by the value matrix yields:

[0044] ,

[0045] in Indicates that the sample borrows corresponding features from other similar samples in the batch. The results of the cross attention operation are weighted summed. Each sample after the cross attention operation is spliced and then linearly transformed to restore it to its original dimension ,Right now , , , for subsequent calculations;

[0046] In multi-head cross-sample attention processing Afterwards, conduct Normalization and residual connection;

[0047] S36, the feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows:

[0048] ,

[0049] in, For the The result of samples being processed by the multi-head cross-sample attention layer, for Normalization processing, The feed-forward neural network (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation.

[0050] The following is the detailed process:

[0051] first, Input into the feedforward neural network layer, in the first layer, the input feature dimension Projected into a higher dimension , usually , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to the original dimension ;

[0052] In feedforward neural network processing Afterwards, the above results were Normalization and residual connection;

[0053] S37, the fertility suitability prediction module is a multi-layer perceptron that receives the features output by the cross-attention feature extraction module between samples. The multilayer perceptron consists of an input layer, five hidden layers, and an output layer. As the input of the multi-layer perceptron, after passing through this seven-layer multi-layer perceptron, two outputs are obtained, namely the probability of being suitable for childbearing and unsuitable for childbearing;

[0054] The calculation process of the entire multilayer perceptron:

[0055] Input layer to 1st hidden layer: ;

[0056] From the 1st hidden layer to the 2nd hidden layer: ;

[0057] 2nd hidden layer to 3rd hidden layer: ;

[0058] 3rd hidden layer to 4th hidden layer: ;

[0059] 4th hidden layer to 5th hidden layer: ;

[0060] Layer 5 to output layer: ,

[0061] in, represents the output from the 1st to the 5th hidden layer, Represent the activation functions ReLU of the 1st to 5th hidden layers respectively, represents the weight matrix from the input layer to the 5th hidden layer, from the 1st to the 5th hidden layer, and from the 5th hidden layer to the output layer, represents the bias vector of each layer, Indicates progress Normalization processing. It represents the output of the fertility suitability prediction module, indicating the probability of being suitable for fertility and unsuitable for fertility.

[0062] Furthermore, the step S4 includes:

[0063] S41. Build a fully supervised prediction network for female patient fertility suitability, use the Adam optimizer, and set the initial learning rate to , set 200 rounds for training, and the learning rate is reduced by 10% every 20 rounds; the input is a row sequence of female patients' reproductive health score data. The text embedding module converts the input data into an embedding vector, and then the embedding vector is input into the cross-attention feature extraction module based on the sample to extract the sample features. The sample features are input into the fertility suitability prediction module for binary classification prediction, and finally the prediction result is output;

[0064] S42. During the training process, the 5-fold cross-validation method will be used to evaluate the performance of the model. Assume that the training set D contains labeled data; then the training set It is evenly divided into 5 subsets, one of which is used as a validation set each time, and the remaining 4 subsets are used for training;

[0065] S43. Use the cross entropy loss function to guide the network training process. The loss function formula is:

[0066] ,

[0067] in, represents the true label, represents the predicted label;

[0068] S44. Train the model using the loss function, adjust the parameters through backpropagation, and select the model parameters with the highest accuracy in the test set for saving;

[0069] S45. Select evaluation indicators including F1 score, ROC curve and AUC value to evaluate the model performance.

[0070] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0071] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0072] It can be seen from the above technical solution that the present invention proposes a method for predicting female fertility suitability based on sample cross-attention. This method cleverly constructs a feature extraction module based on inter-sample cross-attention, and learns different feature information from the sample itself and between samples, thereby effectively improving the overall prediction ability of the model. Before inputting the feature extraction module, the Word2Vec embedding technology is used to convert each row of data in the original table data into an embedding vector, laying a solid foundation for subsequent feature extraction. The fertility suitability prediction module performs a binary classification prediction on the data processed by the feature extraction module. The female patient fertility suitability prediction model based on inter-sample cross-attention has undergone rigorous training and testing, and uses a cross-validation method to demonstrate excellent prediction performance. This method effectively solves the application problem of deep learning in the assessment of female fertility suitability, and provides patients with a more accurate and convenient evaluation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 Schematic diagram of the prediction method steps according to an embodiment of the present invention;

[0074] Figure 2 Schematic diagram of the overall structure of an embodiment of the present invention;

[0075] Figure 3 This is a schematic diagram of an embedded module according to an embodiment of the present invention;

[0076] Figure 4 Schematic diagram of a feature extraction module based on cross-attention between samples according to an embodiment of the present invention;

[0077] Figure 5 Schematic diagram of a fertility suitability prediction module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0079] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a schematic diagram of a method for predicting female fertility suitability based on sample cross attention, and the specific steps are as follows:

[0080] Step 1: Collect a dataset of female physical examination medical records, including medical record text data and labels, divide the dataset into a training set and a test set, and process the training data.

[0081] The specific process is:

[0082] Step 1.1: Collect medical records of women's physical examinations, extract information related to health scores and reproductive health scores, and save them as Word files;

[0083] It should be noted that the female physical examination medical record dataset involved in the embodiments of the present invention has been authorized by the parties concerned, and the relevant data is only used for research purposes.

[0084] Step 1.2: Use the Python library python-docx to extract the health score and reproductive health score from the Word file, and use the Python library openpyxl to save the health score and reproductive health score as an Excel file;

[0085] Step 1.3: Divide the dataset into training data and test data;

[0086] Step 1.4: Use the medical record data as the model input and the output of the fertility suitability prediction module as the final prediction result of the model.

[0087] Step 2: If Figure 3 As shown, a data embedding model based on the word2vec algorithm is constructed, and the generated embedding vector is passed to subsequent structural processing;

[0088] The specific process is:

[0089] Step 2.1: Build a text embedding module based on the word2vec algorithm to convert text into embedding vectors;

[0090] Step 2.2: The text embedding module accepts tabular data as input. ,in To include A tabular dataset of samples, each is a The corresponding eigenvector of It’s a label;

[0091] Step 2.3: Add a classification tag with the learned embedding to each data example .make , is a data point with categorical features;

[0092] Step 2.4: Transform the feature vector Input to an embedding layer In , each feature vector is embedded into d-dimensional space. The embedded feature is represented as ; where input , through the embedding layer Output after .

[0093] Step 3: Construct a cross-attention feature extraction module based on samples to extract features from the embedding vector and pass these features to the fertility suitability prediction module for prediction;

[0094] The specific process is:

[0095] Step 3.1: Construct the feature extraction module based on cross-attention between samples and the fertility suitability prediction module. The feature representation after input embedding based on the cross-attention feature extraction module between samples , output the extracted features ; Input features of the fertility suitability prediction module , output the prediction result;

[0096] Step 3.2: If Figure 4 As shown in Figure 1, the feature extraction module based on cross-attention between samples consists of a multi-head self-attention layer, a multi-head cross-sample attention layer, and two feedforward neural network layers;

[0097] Step 3.3: The multi-head self-attention layer receives the embedded feature representation As input, after multi-head self-attention processing and normalization, the residual connection outputs the result The formula is as follows:

[0098] ,

[0099] in, For the The results of these processing of samples are for Normalization processing, It is multi-head self-attention processing;

[0100] Multi-head self-attention processing : For each sample , each self-attention head has a trainable parameter matrix 、 and , respectively calculated 、 and Get the query matrix , key matrix Sum Matrix . First calculate the dot product , which represents the correlation between different features of the sample. In order to make the gradient of the training process more stable, it is scaled and then Normalization processing, each value becomes a weight coefficient greater than 0 and less than 1, that is, . Multiply the value matrix by the obtained weight coefficient matrix to get:

[0101] ,

[0102] in Indicates the value that needs to be corrected for different features of the sample. The results of multiple self-attention heads are concatenated, that is, . Then perform a linear transformation to restore it to its original dimension ,Right now , , , for subsequent calculations;

[0103] In multi-head self-attention processing Afterwards, the above results conduct Normalization and residual connection;

[0104] Step 3.4: The feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows:

[0105] ,

[0106] in, For the The result of samples processed by the multi-head self-attention layer, for Normalization processing, The feed-forward neural network (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation.

[0107] The following is the detailed process:

[0108] first, Input into the feedforward neural network, in the first layer, the feature dimension of the input Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to the original dimension ;

[0109] In feedforward neural network processing Afterwards, the above results were Normalization and residual connection;

[0110] Step 3.5: The multi-head cross-sample attention layer receives the results of the feedforward neural network layer As input, the obtained results are processed by multi-head cross-sample attention, normalized, and finally the residual connection output is performed. The formula is as follows:

[0111] ,

[0112] in, For the The result of samples processed by the feedforward neural network layer, for Normalization processing, It is a multi-head cross-sample attention process. Unlike self-attention, cross-sample attention is calculated for different data points (i.e. different samples) in a given batch, not just the features of a single data point. Specifically, we concatenate the embedding vector of each feature to a single data point and then calculate the attention value of the sample. This allows us to improve the representation of a given point by examining other points. When some features of a sample are missing or noisy, cross-sample attention can borrow corresponding features from other similar samples in the batch;

[0113] Multi-head cross-sample attention processing :for , randomly select a sample , calculate the cross attention value between the sample and all samples, and then perform Normalization processing, that is ,in express The query matrix, Represents all value matrices. Multiplying the obtained weight coefficient matrix by the value matrix yields:

[0114] ,

[0115] in Indicates that the sample borrows corresponding features from other similar samples in the batch. The results of the cross attention operation are weighted summed. Each sample after the cross attention operation is spliced and then linearly transformed to restore it to its original dimension ,Right now , , , for subsequent calculations;

[0116] In multi-head cross-sample attention processing Afterwards, conduct Normalization and residual connection;

[0117] Step 3.6: The feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows:

[0118] ,

[0119] in, For the The result of samples being processed by the multi-head cross-sample attention layer, for Normalization processing, The feed-forward neural network (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation.

[0120] The following is the detailed process:

[0121] first, Input into the feedforward neural network layer, in the first layer, the input feature dimension Projected into a higher dimension , usually , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to the original dimension ;

[0122] In feedforward neural network processing Afterwards, the above results were Normalization and residual connection;

[0123] Step 3.7: If Figure 5 As shown, the fertility suitability prediction module is a multi-layer perceptron that receives the features output by the cross-attention feature extraction module between samples. The multilayer perceptron consists of an input layer, five hidden layers, and an output layer. As the input of the multi-layer perceptron, after passing through this seven-layer multi-layer perceptron, two outputs are obtained, namely the probability of being suitable for childbearing and unsuitable for childbearing;

[0124] The calculation process of the entire multilayer perceptron:

[0125] Input layer to 1st hidden layer: ;

[0126] From the 1st hidden layer to the 2nd hidden layer: ;

[0127] 2nd hidden layer to 3rd hidden layer: ;

[0128] 3rd hidden layer to 4th hidden layer: ;

[0129] 4th hidden layer to 5th hidden layer: ;

[0130] Layer 5 to output layer: ,

[0131] in, represents the output from the 1st to the 5th hidden layer, Represent the activation functions ReLU of the 1st to 5th hidden layers respectively, represents the weight matrix from the input layer to the 5th hidden layer, from the 1st to the 5th hidden layer, and from the 5th hidden layer to the output layer, represents the bias vector of each layer, Indicates progress Normalization processing. It represents the output of the fertility suitability prediction module, indicating the probability of being suitable for fertility and unsuitable for fertility.

[0132] Step 4: Use the collected training samples to train the above prediction model;

[0133] The specific process is:

[0134] Step 4.1: Build a fully supervised female patient fertility suitability prediction network, use the Adam optimizer, and set the initial learning rate to , set 200 rounds for training, and the learning rate is reduced by 10% every 20 rounds; the input is a row sequence of female patients' reproductive health score data. The text embedding module converts the input data into an embedding vector, and then the embedding vector is input into the cross-attention feature extraction module based on the sample to extract the sample features. The sample features are input into the fertility suitability prediction module for binary classification prediction, and finally the prediction result is output;

[0135] Step 4.2: During the training process, a 5-fold cross-validation method will be used to evaluate the performance of the model; assuming that the training set D contains labeled data; then the training set It is evenly divided into 5 subsets, one of which is used as a validation set each time, and the remaining 4 subsets are used for training;

[0136] Step 4.3: Use the cross entropy loss function to guide the network training process. The loss function formula is:

[0137] ,

[0138] in, represents the true label, represents the predicted label;

[0139] Step 4.4: Train the model using the loss function, adjust the parameters through backpropagation, and select the model parameters with the highest accuracy in the test set and save them;

[0140] Step 4.5: Select evaluation indicators including F1 score, ROC curve and AUC value to evaluate the model performance.

[0141] The following table shows sample data from four female patients and the model's predicted output. We obtained the medical records of these four patients, extracted information related to their health scores and fertility scores, and saved it as data. After data processing, this data was input into our fertility suitability model, which outputs a positive class indicating fertility suitability and a negative class indicating unsuitability.

[0142]

[0143] In summary, the present invention discloses a method for predicting female fertility suitability based on sample cross-attention. The method involves collecting female physical examination medical records, including text data and labels, then constructing a data embedding model based on the word2vec algorithm. The generated embedding vectors are then passed to a feature extraction module based on sample cross-attention. After passing through the feature extraction module, the embedded vectors are converted into contextual features, which are then passed to a fertility suitability prediction module for classification prediction. Finally, the prediction model is trained using the collected training samples.

[0144] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0145] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0146] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the female fertility suitability prediction methods based on sample cross-attention in the above-mentioned embodiments.

[0147] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts of the above method.

[0148] The embodiment of the present application further provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus.

[0149] Memory for storing computer programs;

[0150] The processor is configured to implement the above-mentioned method for segmenting myocardial infarction images without contrast agent when executing the program stored in the memory.

[0151] The communication bus mentioned in the electronic device mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, and a control bus.

[0152] The communication interface is used for communication between the above electronic device and other devices.

[0153] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Alternatively, the memory may be at least one storage device located remotely from the processor.

[0154] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0155] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0156] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0157] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0158] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for predicting female fertility suitability based on sample cross-attention, characterized in that: The following steps are included: S1. Collect a dataset of female medical records, divide the dataset into a training set and a test set as training data, and process the training data; S2. Build a data embedding model based on the word2vec algorithm and generate an embedding vector; S3. Construct a feature extraction module based on cross-attention between samples to extract features from the embedding vector and pass these features to the fertility suitability prediction module for prediction; The inter-sample cross-attention feature extraction module consists of a multi-head self-attention layer, a multi-head cross-sample attention layer and two feedforward neural network layers; The fertility suitability prediction module is a multi-layer perceptron that receives the features output by the cross-attention feature extraction module between samples. is the input; the multilayer perceptron consists of an input layer, five hidden layers and an output layer; the feature As the input of the multi-layer perceptron, after passing through this seven-layer multi-layer perceptron, two outputs are obtained, namely the probability of being suitable for childbearing and unsuitable for childbearing; S4. Using the collected training samples, the above-mentioned fertility suitability prediction module is trained; S5. Inputting the processed medical record data into the trained fertility suitability prediction module to obtain a prediction result; Step S1 includes the following steps: S11. Collect a dataset of female medical examination records, extract information related to health scores and reproductive health scores, and save it as a Word file; S12. Extract the health score and reproductive health score from the Word file using the Python library python-docx, and save the health score and reproductive health score as an Excel file using the Python library openpyxl; S13. Divide the dataset into training data and test data.

2. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S2 includes the following steps: S21. Build a text embedding module based on the word2vec algorithm to convert text into embedding vectors; S22. The text embedding module accepts tabular data as input. ,in To include A tabular dataset of samples, each is a The corresponding eigenvector of It’s a label; S23. Add a classification tag with learned embedding to each data sample ,make , is a data point with categorical features; S24, the feature vector Input to an embedding layer In the , each feature vector is embedded into the d-dimensional space, and the embedded feature is represented as ; where input , through the embedding layer Output after .

3. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S3 includes the following steps: Construct a feature extraction module based on cross-attention between samples and a fertility suitability prediction module, and embed the feature representation based on the input of the feature extraction module based on cross-attention between samples. , output the extracted features ; Input features of the fertility suitability prediction module , output the prediction result; The multi-head self-attention layer receives the embedded feature representation As input, after multi-head self-attention processing and normalization, the residual connection outputs the result ; The formula is as follows: ， in, For the The results of these processing of samples are for Normalization processing, It is multi-head self-attention processing; The feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. ; The formula is as follows: ， in, For the The result of samples processed by the multi-head self-attention layer, for Normalization processing, Processing for feedforward neural networks; The multi-head cross-sample attention layer receives the results of the feedforward neural network layer As input, the obtained results are processed by multi-head cross-sample attention, normalized, and finally the residual connection output is performed. ; The formula is as follows: ， in, For the The result of samples processed by the feedforward neural network layer, for Normalization processing, It is multi-head cross-sample attention processing; The feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. ; The formula is as follows: ， in, For the The result of samples being processed by the multi-head cross-sample attention layer, for Normalization processing, Processing for feedforward neural networks; The calculation process of the entire multi-layer perceptron in the fertility suitability prediction module includes: Input layer to 1st hidden layer: ; From the 1st hidden layer to the 2nd hidden layer: ; 2nd hidden layer to 3rd hidden layer: ; 3rd hidden layer to 4th hidden layer: ; 4th hidden layer to 5th hidden layer: ; Layer 5 to output layer: , in, represents the output from the 1st to the 5th hidden layer, Represent the activation functions ReLU of the 1st to 5th hidden layers respectively, represents the weight matrix from the input layer to the 5th hidden layer, from the 1st to the 5th hidden layer, and from the 5th hidden layer to the output layer, represents the bias vector of each layer, Indicates progress Normalization processing; It represents the output of the fertility suitability prediction module, indicating the probability of being suitable for fertility and unsuitable for fertility.

4. The method for predicting female fertility suitability based on sample cross-attention according to claim 3, characterized in that: Multi-head self-attention processing The following steps are involved: For each sample , each self-attention head has a trainable parameter matrix 、 and , respectively calculated 、 and Get the query matrix , key matrix Sum Matrix ; First calculate the dot product , which represents the correlation between different features of the sample. In order to make the gradient of the training process more stable, it is scaled and then Normalization processing, each value becomes a weight coefficient greater than 0 and less than 1, that is, ; Multiply the obtained weight coefficient matrix by the value matrix to obtain: ， in Indicates the value that needs to be corrected for different features of the sample; the results of multiple self-attention heads are spliced together, that is, ; Then perform a linear transformation to restore it to its original dimension ,Right now , , , for subsequent calculations; In multi-head self-attention processing Afterwards, the above results conduct Normalization and residual connections.

5. The method for predicting female fertility suitability based on sample cross-attention according to claim 3, characterized in that: The feedforward neural network layer in step S34 is a two-layer fully connected neural network, the first layer is a linear transformation and activation function, and the second layer is a linear transformation; Feedforward neural network processing include, first, Input into the feedforward neural network, in the first layer, the feature dimension of the input Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to the original dimension ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connections.

6. The method for predicting female fertility suitability based on sample cross-attention according to claim 3, characterized in that: Multi-head cross-sample attention processing include: for , randomly select a sample , calculate the cross attention value between the sample and all samples, and then Normalization processing, that is ,in express The query matrix, Represents all value matrices; multiplying the value matrix by the obtained weight coefficient matrix yields: ， in Indicates that the sample borrows corresponding features from other similar samples in the batch; The results of the cross attention operation are weighted summed, each sample after the cross attention operation is spliced, and then linearly transformed to restore it to the original dimension ,Right now , , , for subsequent calculations; In multi-head cross-sample attention processing Afterwards, conduct Normalization and residual connections.

7. The method for predicting female fertility suitability based on sample cross-attention according to claim 3, characterized in that: Feedforward neural network processing include, first, Input into the feedforward neural network layer, in the first layer, the input feature dimension Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to the original dimension ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connections.

8. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S4 includes the following steps: S41. Build a fully supervised prediction network for female patient fertility suitability, use the Adam optimizer, and set the initial learning rate to , set 200 rounds for training, and the learning rate is reduced by 10% every 20 rounds; the input is a row sequence of female patients' reproductive health score data. The text embedding module converts the input data into an embedding vector, and then the embedding vector is input into the cross-attention feature extraction module based on the sample to extract the sample features. The sample features are input into the fertility suitability prediction module for binary classification prediction, and finally the prediction result is output; S42. During the training process, the 5-fold cross-validation method will be used to evaluate the performance of the model. Assume that the training set D contains labeled data; then the training set It is evenly divided into 5 subsets, one of which is used as a validation set each time, and the remaining 4 subsets are used for training; S43. Use the cross entropy loss function to guide the network training process. The loss function formula is: ， in, represents the true label, represents the predicted label; S44. Train the model using the loss function, adjust the parameters through backpropagation, and select the model parameters with the highest accuracy in the test set for saving; S45. Select evaluation indicators including F1 score, ROC curve and AUC value to evaluate the model performance.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

Images