Female fertility suitability prediction method based on sample cross attention and medium

Abstract

The invention discloses a female fertility suitability prediction method based on sample cross attention and a medium, and the method comprises the steps: collecting a female physical examination medical record data set, dividing the data set into a training set and a test set as training data, and carrying out the data processing of the training data; constructing a data embedding model based on a word2vec algorithm, and generating an embedding vector; an inter-sample cross attention feature extraction module is constructed to extract features in the embedded vector, and the features are transmitted to a fertility suitability prediction module for prediction; training the fertility suitability prediction module by using the collected training samples; and inputting the processed medical record data into the trained fertility suitability prediction module to obtain a prediction result. According to the method, the problem that a traditional deep learning model does not fully analyze correlation between samples in structured data is solved, accurate prediction of fertility suitability is realized, and effective suggestions are provided for whether women select fertility or not.

Description

Technical Field

[0001] The present invention relates to the field of computer natural language processing, and in particular to a method, device and storage medium for predicting female reproductive suitability based on sample cross-attention. Background Art

[0002] Fertility suitability assessment is essential for family planning and population health. Fertility is not just a physiological process, it also involves great responsibility, especially in raising children. Fertility suitability assessment helps couples make more rational decisions and ensure that they have sufficient resources, abilities and psychological preparation to welcome the arrival of new life. In addition, fertility suitability assessment also includes screening for potential genetic diseases, which is of great significance for protecting family health and avoiding the spread of genetic diseases.

[0003] Traditional methods of assessing fertility suitability usually rely on a series of basic indicators, such as women's age, body mass index (BMI), medical history, pre-pregnancy physical examination data, and genetic screening results. However, these methods have certain limitations, mainly manifested in the inability to comprehensively consider complex factors in multiple dimensions, and the evaluation results are too dependent on the doctor's experience and judgment. Traditional methods have failed to fully utilize the potential of modern medical data and technology, resulting in the inability to provide accurate and personalized evaluation results in some cases.

[0004] With the rapid development of deep learning technology, fertility suitability assessment methods based on deep learning have gradually attracted attention. Deep learning can process more complex multidimensional data and can capture nonlinear relationships between variables, thereby providing more accurate fertility suitability assessments. However, deep learning models are essentially "black box" models. Although they can give high-precision prediction results, their decision-making process is less transparent, which may reduce doctors' and patients' trust in model recommendations. In addition, deep learning models usually require a large amount of high-quality data for training, and these data often involve sensitive personal health information. How to effectively protect data privacy and security has become a major challenge in the application of technology.

[0005] Therefore, the research and application of female fertility suitability prediction methods based on the cross-attention mechanism between samples has important practical significance. By combining deep learning with the cross-attention mechanism between samples, it is possible to better mine the multi-level information in the data, thereby improving the accuracy and credibility of the prediction, while enhancing the interpretability of the model and improving the trust of doctors and patients. Despite this, the current technology still faces many challenges in terms of trust and privacy protection, and has not yet been widely used in clinical practice. How to ensure the accuracy of the technology while improving its transparency and privacy security is a key issue that needs to be solved urgently. Summary of the invention

[0006] The present invention proposes a method, device and storage medium for predicting female reproductive suitability based on sample cross-attention, which can solve at least one of the technical problems in the background technology; specifically, it can solve the problem that the existing technology does not fully utilize deep learning to predict the reproductive suitability of female patients.

[0007] To achieve the above object, the present invention adopts the following technical solutions: A method for predicting female fertility suitability based on sample cross-attention, performing the following steps by a computer device, S1. Collect a dataset of female physical examination medical records, including medical record text data and labels, divide the dataset into a training set and a test set, and process the training data; S2, build a data embedding model based on the word2vec algorithm, and pass the generated embedding vector to subsequent structural processing; S3, construct a feature extraction module based on cross-attention between samples to extract features in the embedding vector, and pass these features to the fertility suitability prediction module for prediction; S4, using the collected training samples to train the above prediction model; S5. When applying, the medical record data after data processing is input into the prediction model to obtain the prediction result.

[0008] Furthermore, the step S1 comprises: S11, collect medical records of female physical examinations, extract information related to health scores and reproductive health scores and save them as word files; S12, using the python library python-docx to extract the health score and reproductive health score from the word file, and using the python library openpyxl to save the health score and reproductive health score as an excel file; S13, dividing the data set into two parts: training data and test data; S14. Use the medical record data as the model input and use the output of the fertility suitability prediction module as the final prediction result of the model.

[0009] Furthermore, the step S2 comprises: S21. Build a text embedding module based on the word2vec algorithm to convert text into embedding vectors; S22. The text embedding module accepts tabular data as input. ,in To include A tabular dataset of samples, each is a The corresponding eigenvector of It’s a label; S23. Add a classification tag with learned embedding to each data sample .make , is a data point with categorical features; S24, the feature vector Input to an embedding layer In , each feature vector is embedded into d-dimensional space. The embedded feature is represented as ; where input , through the embedding layer Output after .

[0010] Furthermore, the step S3 comprises: S31. Construct a feature extraction module based on cross-attention between samples and a fertility suitability prediction module. Feature representation after embedding based on the input of the feature extraction module based on cross-attention between samples , output the extracted features ; Input features of the fertility suitability prediction module , output the prediction result; S32, the feature extraction module based on cross-attention between samples consists of a multi-head self-attention layer, a multi-head cross-sample attention layer and two feed-forward neural network layers; S33, multi-head self-attention layer receives embedded feature representation As input, after multi-head self-attention processing and normalization, the residual connection outputs the result The formula is as follows: , in, For the The results of these processing of samples are for Normalization processing, It is multi-head self-attention processing; Multi-head self-attention processing : For each sample , each self-attention head has a trainable parameter matrix , and , respectively calculate , and Get the query matrix , key matrix Sum Matrix . First, calculate the dot product , which represents the correlation between different features of the sample. In order to make the gradient of the training process more stable, it is scaled and then After normalization, each value becomes a weight coefficient greater than 0 and less than 1, that is, . Multiply the value matrix with the obtained weight coefficient matrix to get: , in Indicates the values ​​that need to be corrected for different features of the sample. The results of multiple self-attention heads are concatenated, that is, Then perform a linear transformation to restore it to its original dimension ,Right now , , , in order to carry out subsequent calculations; In multi-head self-attention processing Afterwards, the above results conduct Normalization and residual connection; S34, the feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows: , in, For the The result of samples processed by multi-head self-attention layers, for Normalization processing, It is processed by feed-forward neural network. The feed-forward neural network layer (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation; The following is the detailed process: first, Input into the feedforward neural network, in the first layer, the input feature dimension Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to original dimensions ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connection; S35, the multi-head cross-sample attention layer receives the results of the feed-forward neural network layer As input, the obtained results are processed by multi-head cross-sample attention, normalized, and finally the residual connection is performed to output the results. The formula is as follows: , in, For the The result of samples being processed by the feed-forward neural network layer. for Normalization processing, It is a multi-head cross-sample attention process. Unlike self-attention, cross-sample attention is calculated for different data points (i.e. different samples) in a given batch, not just the features of a single data point. Specifically, we connect the embedding vector of each feature to a single data point and then calculate the attention value of the sample. This allows us to improve the representation of a given point by checking other points. When some features of a sample are missing or noisy, cross-sample attention can borrow the corresponding features from other similar samples in the batch; For multi-head cross-sample attention processing :for , choose any sample , calculate the cross attention value between this sample and all samples, and then Normalization processing, that is ,in express The query matrix, Represents all value matrices. Multiply the value matrix by the obtained weight coefficient matrix to get: , in Indicates that the sample borrows corresponding features from other similar samples in the batch. The results of the cross-attention operation are weighted summed. Each sample after the cross-attention operation is concatenated and then linearly transformed to restore it to its original dimension. ,Right now , , , in order to carry out subsequent calculations; In multi-head cross-sample attention processing After that, conduct Normalization and residual connection; S36, the feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows: , in, For the The result of samples being processed by a multi-head cross-sample attention layer, for Normalization processing, It is processed by feed-forward neural network. The feed-forward neural network layer (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation; The following is the detailed process: first, Input to the feedforward neural network layer, in the first layer, the input feature dimension Projected into a higher dimension , usually , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to original dimensions ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connection; S37, the fertility suitability prediction module is a multi-layer perceptron that receives the features output by the cross-attention feature extraction module based on the samples. The multilayer perceptron consists of an input layer, five hidden layers, and an output layer. As the input of the multi-layer perceptron, two outputs are obtained after passing through this seven-layer multi-layer perceptron, namely the probability of being suitable for childbearing and unsuitable for childbearing; The calculation process of the entire multilayer perceptron is: Input layer to 1st hidden layer: ; From the 1st hidden layer to the 2nd hidden layer: ; From the 2nd hidden layer to the 3rd hidden layer: ; 3rd hidden layer to 4th hidden layer: ; 4th hidden layer to 5th hidden layer: ; Layer 5 to output layer: , in, represents the output from the 1st to the 5th hidden layer, Represent the activation functions ReLU of the 1st to 5th hidden layers respectively, represents the weight matrix from the input layer to the 5th hidden layer, from the 1st to the 5th hidden layer, and from the 5th hidden layer to the output layer, represents the bias vector of each layer, Indicates that Normalization processing. Represents the output of the fertility suitability prediction module, indicating the probability of being suitable for fertility and unsuitable for fertility.

[0011] Furthermore, the step S4 comprises: S41. Build a fully supervised prediction network for fertility suitability of female patients, use the Adam optimizer, and set the initial learning rate to , set 200 rounds for training, and the learning rate is reduced by 10% every 20 rounds; the input is a row sequence of female patients' reproductive health score data, and the text embedding module converts the input data into an embedding vector, and then the embedding vector is input into the feature extraction module based on cross-attention between samples to extract the features of the samples, and the sample features are input into the fertility suitability prediction module for binary classification prediction, and finally the prediction results are output; S42. During the training process, a 5-fold cross-validation method will be used to evaluate the performance of the model. Assume that the training set D contains labeled data; then the training set It is evenly divided into 5 subsets, one of which is used as a validation set each time, and the remaining 4 subsets are used for training; S43, using the cross entropy loss function to guide the network training process, the loss function formula is: , in, represents the true label, represents the predicted label; S44, training the model through the loss function, adjusting the parameters through back propagation, and selecting the model parameters with the highest accuracy in the test set for saving; S45. Select evaluation indicators including F1 score, ROC curve and AUC value to evaluate the model performance.

[0012] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0013] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0014] It can be seen from the above technical solution that the present invention proposes a method for predicting female fertility suitability based on sample cross-attention. This method cleverly constructs a feature extraction module based on cross-attention between samples, and learns different feature information from the sample itself and between samples, thereby effectively improving the overall prediction ability of the model. Before inputting the feature extraction module, the Word2Vec embedding technology is used to convert each row of data in the original table data into an embedded vector, laying a solid foundation for subsequent feature extraction. The fertility suitability prediction module performs binary classification prediction on the data processed by the feature extraction module. The female patient fertility suitability prediction model based on cross-attention between samples has been rigorously trained and tested, and uses a cross-validation method to demonstrate excellent prediction performance. This method effectively solves the application problem of deep learning in the assessment of female fertility suitability, and provides patients with a more accurate and convenient means of assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the steps of the prediction method according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the overall structure of an embodiment of the present invention; Figure 3 This is a schematic diagram of an embedded module according to an embodiment of the present invention; Figure 4 It is a schematic diagram of a feature extraction module based on cross-attention among samples according to an embodiment of the present invention; Figure 5 Schematic diagram of a fertility suitability prediction module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0017] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a schematic diagram of a method for predicting female fertility suitability based on sample cross-attention, and the specific steps are as follows: Step 1: Collect a dataset of female physical examination medical records, including medical record text data and labels, divide the dataset into a training set and a test set, and process the training data.

[0018] The specific process is: Step 1.1: Collect medical records of women’s physical examinations, extract information about health scores and reproductive health scores, and save them as word files; Step 1.2: Use the python library python-docx to extract the health score and reproductive health score in the word file, and use the python library openpyxl to save the health score and reproductive health score as an excel file; Step 1.3: Divide the dataset into training data and test data; Step 1.4: Use the medical record data as the model input and the output of the fertility suitability prediction module as the final prediction result of the model.

[0019] Step 2: If Figure 3 As shown, a data embedding model based on the word2vec algorithm is constructed, and the generated embedding vector is passed to subsequent structural processing; The specific process is: Step 2.1: Build a text embedding module based on the word2vec algorithm to convert text into embedding vectors; Step 2.2: The text embedding module accepts tabular data as input. Let ,in To include A tabular dataset of samples, each is a The corresponding eigenvector of It’s a label; Step 2.3: Add a classification tag with the learned embedding to each data sample .make , is a data point with categorical features; Step 2.4: Transform the feature vector Input to an embedding layer In , each feature vector is embedded into d-dimensional space. The embedded feature is represented as ; where input , through the embedding layer Output after .

[0020] Step 3: Construct a feature extraction module based on cross-attention between samples to extract features from the embedding vector, and pass these features to the fertility suitability prediction module for prediction; The specific process is: Step 3.1: Construct the feature extraction module based on cross-attention between samples and the fertility suitability prediction module. Feature representation after embedding based on the input of the feature extraction module based on cross-attention between samples , output the extracted features ; Input features of the fertility suitability prediction module , output the prediction result; Step 3.2: If Figure 4 As shown, the feature extraction module based on cross-attention between samples consists of a multi-head self-attention layer, a multi-head cross-sample attention layer and two feed-forward neural network layers; Step 3.3: The multi-head self-attention layer receives the embedded feature representation As input, after multi-head self-attention processing and normalization, the residual connection outputs the result The formula is as follows: , in, For the The results of these processing of samples are for Normalization processing, It is multi-head self-attention processing; Multi-head self-attention processing : For each sample , each self-attention head has a trainable parameter matrix , and , respectively calculated , and Get the query matrix , key matrix Sum Matrix . First, calculate the dot product , which represents the correlation between different features of the sample. In order to make the gradient of the training process more stable, it is scaled and then After normalization, each value becomes a weight coefficient greater than 0 and less than 1, that is, . Multiply the value matrix with the obtained weight coefficient matrix to get: , in Indicates the values ​​that need to be corrected for different features of the sample. The results of multiple self-attention heads are concatenated, that is, Then perform a linear transformation to restore it to its original dimension ,Right now , , , in order to carry out subsequent calculations; In multi-head self-attention processing Afterwards, the above results conduct Normalization and residual connection; Step 3.4: The feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows: , in, For the The result of samples processed by multi-head self-attention layers, for Normalization processing, It is processed by feed-forward neural network. The feed-forward neural network layer (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation; The following is the detailed process: first, Input into the feedforward neural network, in the first layer, the input feature dimension Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to original dimensions ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connection; Step 3.5: The multi-head cross-sample attention layer receives the results of the feed-forward neural network layer As input, the obtained results are processed by multi-head cross-sample attention, normalized, and finally the residual connection is performed to output the results. The formula is as follows: , in, For the The result of samples being processed by the feed-forward neural network layer. for Normalization processing, It is a multi-head cross-sample attention process. Unlike self-attention, cross-sample attention is calculated for different data points (i.e. different samples) in a given batch, not just the features of a single data point. Specifically, we connect the embedding vector of each feature to a single data point and then calculate the attention value of the sample. This allows us to improve the representation of a given point by checking other points. When some features of a sample are missing or noisy, cross-sample attention can borrow the corresponding features from other similar samples in the batch; For multi-head cross-sample attention processing :for , choose any sample , calculate the cross attention value between this sample and all samples, and then Normalization processing, that is ,in express The query matrix, Represents all value matrices. Multiply the value matrix by the obtained weight coefficient matrix to get: , in Indicates that the sample borrows corresponding features from other similar samples in the batch. The results of the cross-attention operation are weighted summed. Each sample after the cross-attention operation is concatenated and then linearly transformed to restore it to its original dimension. ,Right now , , , in order to carry out subsequent calculations; In multi-head cross-sample attention processing After that, conduct Normalization and residual connection; Step 3.6: The feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. The formula is as follows: , in, For the The result of samples being processed by a multi-head cross-sample attention layer, for Normalization processing, It is processed by feed-forward neural network. The feed-forward neural network layer (FFN) is a two-layer fully connected neural network. The first layer is a linear transformation and activation function, and the second layer is a linear transformation; The following is the detailed process: first, Input to the feedforward neural network layer, in the first layer, the input feature dimension Projected into a higher dimension , usually , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to original dimensions ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connection; Step 3.7: If Figure 5 As shown in the figure, the fertility suitability prediction module is a multi-layer perceptron that receives the features output by the cross-attention feature extraction module between samples. The multilayer perceptron consists of an input layer, five hidden layers, and an output layer. As the input of the multi-layer perceptron, two outputs are obtained after passing through this seven-layer multi-layer perceptron, namely the probability of being suitable for childbearing and unsuitable for childbearing; The calculation process of the entire multilayer perceptron is: Input layer to 1st hidden layer: ; From the 1st hidden layer to the 2nd hidden layer: ; From the 2nd hidden layer to the 3rd hidden layer: ; 3rd hidden layer to 4th hidden layer: ; 4th hidden layer to 5th hidden layer: ; Layer 5 to output layer: , in, represents the output from the 1st to the 5th hidden layer, Represent the activation functions ReLU of the 1st to 5th hidden layers respectively, represents the weight matrix from the input layer to the 5th hidden layer, from the 1st to the 5th hidden layer, and from the 5th hidden layer to the output layer, represents the bias vector of each layer, Indicates Normalization processing. Represents the output of the fertility suitability prediction module, indicating the probability of being suitable for fertility and unsuitable for fertility.

[0021] Step 4: Use the collected training samples to train the above prediction model; The specific process is: Step 4.1: Construct a fully supervised female patient fertility suitability prediction network, use the Adam optimizer, and set the initial learning rate to , set 200 rounds for training, and the learning rate is reduced by 10% every 20 rounds; the input is a row sequence of female patients' reproductive health score data, and the text embedding module converts the input data into an embedding vector, and then the embedding vector is input into the feature extraction module based on cross-attention between samples to extract the features of the samples, and the sample features are input into the fertility suitability prediction module for binary classification prediction, and finally the prediction results are output; Step 4.2: During the training process, a 5-fold cross-validation method will be used to evaluate the performance of the model; assuming that the training set D contains labeled data; then the training set It is evenly divided into 5 subsets, one of which is used as a validation set each time, and the remaining 4 subsets are used for training; Step 4.3: Use the cross entropy loss function to guide the network training process. The loss function formula is: , in, represents the true label, represents the predicted label; Step 4.4: Train the model through the loss function, adjust the parameters through back propagation, and select the model parameters with the highest accuracy in the test set for saving; Step 4.5: Select evaluation indicators including F1 score, ROC curve and AUC value to evaluate the model performance.

[0022] As shown in the table below, this is the sample data of four female patients and the prediction of the model output. We obtained the medical examination records of the four patients, extracted the information about health scores and fertility health scores and saved them as data information. After data processing, it was input into our fertility suitability model. The output of positive class indicates suitable for childbearing, and negative class indicates unsuitable for childbearing.

[0023]

[0024] In general, the present invention discloses a method for predicting female fertility suitability based on sample cross-attention. The method comprises collecting female physical examination medical records, including medical record text data and labels, then constructing a data embedding model based on the word2vec algorithm, and passing the generated embedding vector to a feature extraction module based on sample cross-attention, the embedding vector is converted into context features after passing through the feature extraction module based on sample cross-attention, and these features are passed to a fertility suitability prediction module for classification prediction. Finally, the above prediction model is trained using the collected training samples.

[0025] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0026] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0027] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the female fertility suitability prediction methods based on sample cross-attention in the above-mentioned embodiments.

[0028] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above method.

[0029] The embodiment of the present application also provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. Memory, used to store computer programs; The processor is used to implement the above-mentioned contrast-free myocardial infarction image segmentation method when executing the program stored in the memory.

[0030] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.

[0031] The communication interface is used for communication between the above electronic device and other devices.

[0032] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0033] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0034] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0035] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises a..." does not exclude the existence of other identical elements in the process, method, article or device that includes the elements. Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0036] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting female fertility suitability based on sample cross-attention, characterized in that: The following steps are included: S1. Collect a dataset of female physical examination medical records, divide the dataset into a training set and a test set as training data, and perform data processing on the training data; S2. Build a data embedding model based on the word2vec algorithm and generate an embedding vector; S3, construct a feature extraction module based on cross-attention between samples to extract features in the embedding vector, and pass these features to the fertility suitability prediction module for prediction; S4, using the collected training samples to train the above-mentioned fertility suitability prediction module; S5. Input the processed medical record data into the trained fertility suitability prediction module to obtain the prediction result.

2. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S1 includes the following steps: S11, collect female physical examination medical record data sets, extract information related to health scores and reproductive health scores and save them as word files; S12, using the python library python-docx to extract the health score and reproductive health score from the word file, and using the python library openpyxl to save the health score and reproductive health score as an excel file; S13. Divide the data set into two parts: training data and test data.

3. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S2 includes the following steps: S21. Build a text embedding module based on the word2vec algorithm to convert text into embedding vectors; S22. The text embedding module accepts tabular data as input. ,in To include A tabular dataset of samples, each is a The corresponding eigenvector of It’s a label; S23. Add a classification tag with learned embedding to each data sample ,make , is a data point with categorical features; S24, the feature vector Input to an embedding layer In the above example, each feature vector is embedded into the d-dimensional space, and the embedded feature is represented as ; where input , through the embedding layer Output after .

4. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S3 includes the following steps: S31. Construct a feature extraction module based on cross-attention between samples and a fertility suitability prediction module, and embed the feature representation after inputting the cross-attention feature extraction module between samples. , output the extracted features ; Input features of the fertility suitability prediction module , output the prediction result; S32, the feature extraction module based on cross-attention between samples consists of a multi-head self-attention layer, a multi-head cross-sample attention layer and two feed-forward neural network layers; S33, multi-head self-attention layer receives embedded feature representation As input, after multi-head self-attention processing and normalization, the residual connection outputs the result ; The formula is as follows: ， in, For the The results of these processing of samples are for Normalization processing, It is multi-head self-attention processing; S34, the feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. ; The formula is as follows: ， in, For the The result of samples processed by multi-head self-attention layers, for Normalization processing, Processing for feed-forward neural networks; S35, the multi-head cross-sample attention layer receives the results of the feed-forward neural network layer As input, the obtained results are processed by multi-head cross-sample attention, normalized, and finally the residual connection is performed to output the results. ; The formula is as follows: ， in, For the The result of samples being processed by the feed-forward neural network layer. for Normalization processing, It is multi-head cross-sample attention processing; S36, the feedforward neural network layer receives the results of the multi-head self-attention layer As input, the result is processed by feedforward neural network, normalized, and finally the residual connection is used to output the result. ; The formula is as follows: ， in, For the The result of samples being processed by a multi-head cross-sample attention layer, for Normalization processing, Processing for feed-forward neural networks; S37, the fertility suitability prediction module is a multi-layer perceptron that receives the features output by the cross-attention feature extraction module based on the samples. is the input; the multilayer perceptron consists of an input layer, five hidden layers and an output layer; As the input of the multi-layer perceptron, two outputs are obtained after passing through this seven-layer multi-layer perceptron, namely the probability of being suitable for childbearing and unsuitable for childbearing; The calculation process of the entire multilayer perceptron is: Input layer to 1st hidden layer: ; From the 1st hidden layer to the 2nd hidden layer: ; 2nd hidden layer to 3rd hidden layer: ; 3rd hidden layer to 4th hidden layer: ; 4th hidden layer to 5th hidden layer: ; Layer 5 to output layer: , in, represents the output from the 1st to the 5th hidden layer, Represent the activation functions ReLU of the 1st to 5th hidden layers respectively, represents the weight matrix from the input layer to the 5th hidden layer, from the 1st to the 5th hidden layer, and from the 5th hidden layer to the output layer, represents the bias vector of each layer, Indicates that Normalization processing; Represents the output of the fertility suitability prediction module, indicating the probability of being suitable for fertility and unsuitable for fertility.

5. The method for predicting female fertility suitability based on sample cross-attention according to claim 4 is characterized in that: Multi-head self-attention processing The following steps are involved: For each sample , each self-attention head has a trainable parameter matrix , and , respectively calculate , and Get the query matrix , key matrix Sum Matrix ; First calculate the dot product , which represents the correlation between different features of the sample. In order to make the gradient of the training process more stable, it is scaled and then After normalization, each value becomes a weight coefficient greater than 0 and less than 1, that is, ; Multiply the value matrix with the obtained weight coefficient matrix to obtain: ， in Indicates the value that needs to be corrected for different features of the sample; concatenate the results of multiple self-attention heads, that is ; Then perform a linear transformation to restore it to its original dimension ,Right now , , , in order to carry out subsequent calculations; In multi-head self-attention processing Afterwards, the above results conduct Normalization and residual connections.

6. The method for predicting female fertility suitability based on sample cross-attention according to claim 4, characterized in that: The feedforward neural network layer in step S34 is a two-layer fully connected neural network, the first layer is a linear transformation and activation function, and the second layer is a linear transformation; Feedforward Neural Network Processing include, first, Input into the feedforward neural network, in the first layer, the input feature dimension Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to original dimensions ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connections.

7. The method for predicting female fertility suitability based on sample cross-attention according to claim 4, characterized in that: Multi-head cross-sample attention processing include: for , choose any sample , calculate the cross attention value between this sample and all samples, and then Normalization processing, that is ,in express The query matrix, Represents all value matrices; multiply the value matrix by the obtained weight coefficient matrix to obtain: ， in Indicates that the sample borrows corresponding features from other similar samples in the batch; The results of the cross-attention operation are weighted summed, each sample after the cross-attention operation is concatenated, and then linearly transformed to restore it to the original dimension ,Right now , , , in order to carry out subsequent calculations; In multi-head cross-sample attention processing After that, conduct Normalization and residual connections.

8. The method for predicting female fertility suitability based on sample cross-attention according to claim 4, characterized in that: Feedforward Neural Network Processing include, first, Input to the feedforward neural network layer, in the first layer, the input feature dimension Projected into a higher dimension ,Right now , and then use the activation function ReLU to increase the nonlinear expression ability of the feature; in the second layer, the high-dimensional features Project back to original dimensions ; In feedforward neural network processing Afterwards, the above results were Normalization and residual connections.

9. The method for predicting female fertility suitability based on sample cross-attention according to claim 1, characterized in that: Step S4 includes the following steps: S41. Build a fully supervised prediction network for fertility suitability of female patients, use the Adam optimizer, and set the initial learning rate to , set 200 rounds for training, and the learning rate is reduced by 10% every 20 rounds; the input is a row sequence of female patients' reproductive health score data, and the text embedding module converts the input data into an embedding vector, and then the embedding vector is input into the feature extraction module based on cross-attention between samples to extract the features of the samples, and the sample features are input into the fertility suitability prediction module for binary classification prediction, and finally the prediction results are output; S42. During the training process, a 5-fold cross-validation method will be used to evaluate the performance of the model. Assume that the training set D contains labeled data; then the training set It is evenly divided into 5 subsets, one of which is used as a validation set each time, and the remaining 4 subsets are used for training; S43, using the cross entropy loss function to guide the network training process, the loss function formula is: ， in, represents the true label, represents the predicted label; S44, training the model through the loss function, adjusting the parameters through back propagation, and selecting the model parameters with the highest accuracy in the test set for saving; S45. Select evaluation indicators including F1 score, ROC curve and AUC value to evaluate the model performance.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 9.

Images