Overlapped blastosphere target detection method based on multi-dimensional feature fusion

Abstract

The invention discloses an overlapped blastosphere target detection method based on multi-dimensional feature fusion, and the method comprises the steps: 1, obtaining embryo images in 1-8 cell periods to construct a data set, and carrying out the preprocessing to construct a sample set; 2, separating a phase component and an amplitude component by randomly selecting 1-2 cell images in a training set, and performing phase fusion on the images; 3, performing inverse Fourier transform on the fused phase and amplitude to obtain a new training sample set; 4, constructing a network model for cleavage sphere target detection; and 5, carrying out cleavage ball detection by utilizing the trained model. According to the method, data enhancement is carried out by introducing interference to phase information of an image, and meanwhile, fine-grained feature extraction and multi-scale global feature aggregation are carried out through multi-dimensional feature fusion (weighting of height, width and channels), so that the separation capability of overlapped regions is improved, and the accuracy of overlapped cleavage sphere detection can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of blastomere target detection, and in particular to an image preprocessing method based on phase fusion and an improved method for an overlapping blastomere detection model based on multi-dimensional feature fusion. Background Art

[0002] The blastomere detection technology during the embryonic cell development period plays a vital role in the early human life development process. This technology can not only identify whether there are abnormal divisions or non-divisions during embryonic development by analyzing the number of blastomeres, but also use the relevant information of blastomeres to trace the newly generated blastomeres in the subsequent division process and other pathological analyses. Therefore, whether the blastomeres can be accurately and effectively detected and identified has become one of the key technologies in this field. In addition, this is also an indispensable part of embryo evaluation, which is of great significance for helping embryologists evaluate the quality of embryos and their development potential. According to the D3 embryo evaluation system, on the 1st to 3rd day after embryo formation (ie D1-D3), doctors can obtain key morphological characteristics of the embryo, including important parameters such as the number and morphology of blastomeres. These data are one of the important bases for judging the quality of embryos.

[0003] In the medical field, especially in embryology research, manual identification of a large number of images is a time-consuming and professional task. Therefore, using computer technology to build a visual model to assist doctors in quickly and accurately detecting blastomeres has a profound impact on this research field. This technology can greatly improve work efficiency, reduce the workload of doctors, and improve the accuracy of diagnosis.

[0004] The current mainstream blastomere detection methods mainly rely on traditional schemes such as ellipse detection and arc segment detection. Although these methods have faster processing speed and lower computing resource usage, they are often unsatisfactory in practical applications. Especially in the 4-cell stage and above, due to the increase in the number of blastomeres and the high degree of overlap, the limitations of traditional detection schemes are more obvious.

[0005] In contrast, deep learning algorithms have been widely used in various target detection tasks and have achieved remarkable results due to their excellent generalization ability and powerful automatic feature learning ability. However, in the field of blastomere detection, deep learning algorithms face unique difficulties in dealing with the challenge of overlapping blastomeres. As the number of blastomeres increases, the overlap between blastomeres becomes more significant, and the morphological overlap of blastomeres also shows diversity, which brings greater challenges to traditional detection algorithms. In addition, due to the translucent nature of the blastomeres themselves, the overlap between blastomeres is unique compared to the complete occlusion caused by the overlap of other scenes. This makes it difficult for current detection algorithms to adapt to scenes with highly overlapping blastomeres, and their detection results are often unsatisfactory.

[0006] First, the increase in blastomere overlap makes feature extraction and boundary detection more complicated, and traditional feature representation and detection methods are difficult to effectively deal with this situation. Second, due to the diversity of blastomere morphological overlap, existing algorithms have certain difficulties in capturing and distinguishing this diversity. Finally, existing algorithms may lack robustness to the situation of highly overlapping blastomeres and cannot accurately identify and locate overlapping blastomeres. Summary of the invention

[0007] The present invention aims to address the deficiencies in the prior art and proposes an overlapping blastomere target detection method based on multi-dimensional feature fusion, in order to solve the problem of low accuracy in detecting blastomeres in a highly overlapping environment, enhance the generalization of the model to deal with blastomere occlusion, thereby maintaining a high detection efficiency while improving detection accuracy, and being more conducive to the application of blastomere detection in highly overlapping scenarios.

[0008] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0009] The overlapping blastomere target detection method based on multi-dimensional feature fusion of the present invention is characterized in that it is carried out according to the following steps:

[0010] Step 1: Obtain the embryo image dataset of the 1-8 cell period and perform preprocessing to obtain the embryo image sample set , represents the i-th embryo image sample, N represents the total number of embryo image samples, The blastomere region divided in the figure is marked to obtain The label, The real position label frame of the jth blastomere is recorded as ,in, , They are The coordinates of the upper left corner and upper right corner of the j-th blastomere, , are the coordinates of the lower left corner and lower right corner of the jth blastomere respectively;

[0011] Step 2: Perform Fourier transform and get Phase With amplitude , and then use formula (1) to get New phase of :

[0012] (1)

[0013] In formula (1), Represents the weight value, Represents the jth embryo image sample The phase of

[0014] Step 3: New phase of With amplitude Perform inverse Fourier transform to obtain the new i-th embryo image sample , thus forming a new embryo image sample set ;

[0015] Step 4: Build a blastomere target detection network, including: backbone network, neck network and head network;

[0016] Step 4.1: The backbone network Processing is performed to obtain the corresponding blastomere deep fusion convolution feature map ;

[0017] Step 4.2: The neck network Processing is performed to obtain the global feature map of the i-th blastomere ;

[0018] Step 4.3, the head network includes: M convolutional layers, H pooling layers, and a prediction head;

[0019] Input into the head network, and after feature extraction through M convolutional layers, the convolution feature map of the i-th blastomere is obtained. ;

[0020] After downsampling in H pooling layers, the downsampling feature map of the i-th blastomere is obtained. ;

[0021] Predict Head Pair After processing, we get The predicted coordinates of the j-th blastomere bounding box and the predicted class probabilities of blastomere targets ;in, They are The predicted coordinates of the upper left corner and the upper right corner of the j-th blastomere, , They are The predicted coordinates of the lower left corner and the lower right corner of the j-th blastomere;

[0022] Step 4.4: Construct the total loss of the blastomere target detection network ;

[0023] Step 5: Based on , use the gradient descent method to train the blastomere target detection network and calculate the total loss To update the network parameters until the total loss After convergence or reaching the maximum number of iterations, the training is stopped to generate a blastomere detection model corresponding to the optimal parameters, which is used to detect blastomeres in the test set embryo image sample set.

[0024] The overlapping blastomere target detection method based on multi-dimensional feature fusion described in the present invention is also characterized in that the backbone network in step 4.1 includes: a first convolution module, S enhanced convolution feature fusion modules, and P enhanced downsampling feature fusion modules in sequence; wherein each enhanced convolution feature fusion module includes: a convolution module and a C2f_RFMGConv module, and each enhanced downsampling feature fusion module includes: a downsampling module and a C2f_RFMGConv module;

[0025] Step 4.1.1 The first convolution module of the input backbone is used for feature extraction to generate Initial characteristics of blastomere ;

[0026] Step 4.1.2, when s=1, Input into the sth enhanced convolution feature fusion module and after being processed by a convolution module, we get Standard convolution feature map of the sth blastomere After being processed by a C2f_RFMGConv module, the standard fusion feature map of the sth blastomere is obtained. ;

[0027] Step 4.1.3, when p=1, Input into the pth enhanced downsampling feature fusion module, and after being processed by a downsampling module, the pth blastomere convolution downsampling feature map is obtained. After being processed by a C2f_RFMGConv module, the pth blastomere fusion feature map is obtained. ;

[0028] When p=2,3,…,P, fuse the p-1th blastomere with characteristic graphs The input is processed in the p-th enhanced downsampling feature fusion module, so that the P-th enhanced downsampling feature fusion module outputs the final P-th blastomere deep fusion convolution feature map. .

[0029] Furthermore, the C2f_RFMGConv module in step 4.1.2 is composed of the second convolution module, the feature map splitting module, the Bottleneck_RFMGConv module, and the third convolution module in sequence;

[0030] Step 4.1.2.1. After processing by the second convolution module, the advanced convolution feature map of the sth blastomere is obtained. ;

[0031] Step 4.1.2.2 The input feature map is processed in the splitting module to obtain the feature map before advanced convolution of the sth blastomere And the feature map after advanced convolution of the sth blastomere ;

[0032] Step 4.1.2.3: The convolution module in the Bottleneck_RFMGConv module Perform feature extraction to obtain the deep convolution feature map of the sth blastomere ;

[0033] The multi-dimensional feature fusion module in the Bottleneck_RFMGConv module uses formula (2) to After feature generation processing, the global feature map of the sth blastomere is obtained , then After feature rearrangement processing, the sth blastomere rearrangement feature map is obtained :

[0034] (2)

[0035] In formula (2), is the activation function, For normalization, is a two-dimensional convolution operation, is the weight tensor of the convolution kernel in the multi-dimensional feature fusion module, is the stride of the convolution operation;

[0036] Using formula (3), we can get the s-th highly pooled features: , the s-th width pooling feature And the sth global average pooling feature , and then use formula (4) to get the characteristic map of the sth blastomere fusion :

[0037] (3)

[0038] (4)

[0039] In formula (3), is a two-dimensional adaptive average pooling operation, The height dimension of the pooled output size is h, and the width dimension is 1; h represents the height of the input feature map, and w represents the width of the input feature map. Represents the weight tensor of the convolution kernel in the average pooling module; Indicates that the input feature map is at position The eigenvalue at ;

[0040] In formula (4), , , Respectively represent the weight tensors in the convolution kernels used in the corresponding height pooling, width pooling, and global average pooling operations;

[0041] Step 4.1.2.4 After the feature extraction process of the third convolution module, the standard fusion feature map of the sth blastomere is obtained. .

[0042] Furthermore, the neck network in step 4.2 includes, in sequence: L enhanced upsampling feature fusion modules, a fourth convolution module, a first C2f_RFMGConv module, and a C2fCIB module; wherein each enhanced upsampling feature fusion module includes: an upsampling module and a C2f_RFMGConv module;

[0043] Step 4.2.1: hour, Input neck network In the enhanced upsampling feature fusion module, and after being processed by an upsampling module, the first Blastomere upsampling convolution feature map ,Will Enter to The enhanced upsampling feature fusion module is processed in the C2f_RFMGConv module to obtain the Characteristic diagram of deep fusion of blastomeres ;

[0044] when =2,3,…,L, Characteristic diagram of deep fusion of blastomeres Enter the The Lth enhanced upsampling feature fusion module is used for processing, so that the final Lth blastomere fusion feature map is output by the Lth enhanced upsampling feature fusion module. ;

[0045] Step 4.2.2 Input into the fourth convolution module for feature extraction to obtain the feature map of the i-th blastomere ;

[0046] Step 4.2.3 Input into the first C2f_RFMGConv module for processing to obtain the feature map after the fusion of the i-th blastomere ;

[0047] Step 4.2.3 After being input into the C2fCIB module for processing, the global feature map of the i-th blastomere is obtained. .

[0048] Further, the step 4.4 includes:

[0049] Step 4.4.1: Use formula (5) to construct the bounding box loss :

[0050] (5)

[0051] In formula (5), J represents The total number of blastomeres predicted in ;

[0052] Step 4.4.2: Use formula (6) to construct the cross entropy loss :

[0053] (6)

[0054] Step 4.4.3: Use formula (7) to construct the total loss :

[0055] (7).

[0056] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the overlapping blastomere target detection method, and the processor is configured to execute the program stored in the memory.

[0057] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the overlapping blastomere target detection method when the computer program is executed by a processor.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The present invention adopts the phase fusion method to enhance the data of the training set, and simulates the occlusion interference between blastomeres in the multicellular period through phase fusion, which greatly enriches the data set without changing the network complexity and improves the generalization of the blastomere detection model;

[0060] 2. The present invention proposes a C2f_RFMGConv module, which retains and highlights the details of specific locations through multi-dimensional feature fusion (weighted height, width and channel). In the overlapping area of ​​blastomeres, the module captures slight differences through a fine-grained feature generation mechanism, can extract local and global features with higher resolution, and the refined feature extraction process makes the generated feature weights more accurate, which helps to suppress background interference, thereby improving the separation ability of the blastomere detection network for overlapping areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flow chart of a training method for an overlapping blastomere detection model according to an embodiment of the present invention;

[0062] Figure 2 is a flow chart of a phase fusion algorithm according to an embodiment of the present invention;

[0063] Figure 3 is a structural diagram of an overlapping blastomere detection model according to an embodiment of the present invention;

[0064] Figure 4 is a structural diagram of a C2f_RFMGConv module according to an embodiment of the present invention;

[0065] Figure 5 4 is a diagram showing the effect of overlapping blastomere detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In this embodiment, a method for detecting overlapping blastomere targets based on multi-dimensional feature fusion is provided. Figure 1 As shown, the following steps are included:

[0067] Step 1: Obtain the embryo image dataset of the 1-8 cell period and perform preprocessing to obtain the embryo image sample set , represents the i-th embryo image sample, N represents the total number of embryo image samples, The blastomere region divided in the figure is marked to obtain The label, The real position label frame of the jth blastomere is recorded as ,in, , They are The coordinates of the upper left corner and upper right corner of the j-th blastomere, , are the lower left corner coordinates and lower right corner coordinates of the jth blastomere, respectively; in this embodiment, samples collected in the time difference incubator of Chongqing Maternal and Child Health Hospital are used, the training set contains images of the 1-4 cell period, the test set contains images of the 1-8 cell period, and the image size is 800×800.

[0068] Step 2: Figure 2 As shown, the training sample set images of 1-2 cell period are selected ,right Perform Fourier transform and get Phase With amplitude , and then use formula (1) to get New phase of :

[0069] (1)

[0070] in Represents the weight value, Represents the jth embryo image sample In this embodiment, only 50% of the 1-2 cell training set images are selected for data enhancement of phase fusion, where the weight value is a random value between (0.01~0.2).

[0071] Step 3: New phase of With amplitude Perform inverse Fourier transform to obtain the new i-th embryo image sample , thus forming a new embryo image sample set ;

[0072] Step 4: Build a blastomere target detection network, including: backbone network, neck network and head network, such as Figure 3 As shown;

[0073] Step 4.1, the backbone network includes in sequence: a first convolution module, S enhanced convolution feature fusion modules, and P enhanced downsampling feature fusion modules; wherein each enhanced convolution feature fusion module includes: a convolution module and a C2f_RFMGConv module, and each enhanced downsampling feature fusion module includes: a downsampling module and a C2f_RFMGConv module; in this embodiment, 2 enhanced convolution feature fusion modules and 2 enhanced downsampling feature fusion modules are included.

[0074] Step 4.1.1 The first convolution module of the input backbone is used for feature extraction to generate Initial characteristics of blastomere ; In this embodiment, the size of the convolution kernel is 3×3, and the stride is 2;

[0075] Step 4.1.2, when s=1, Input into the sth enhanced convolution feature fusion module and after being processed by a convolution module, we get Standard convolution feature map of the sth blastomere After being processed by a C2f_RFMGConv module, the standard fusion feature map of the sth blastomere is obtained. ;

[0076] like Figure 4 As shown in FIG. 1 , the C2f_RFMGConv module is composed of the second convolution module, the feature map splitting module, the Bottleneck_RFMGConv module, and the third convolution module in sequence; in this embodiment, the Bottleneck_RFMGConv module introduces a gating mechanism, and when the input is True, a residual connection is introduced; if the input is False, no residual is introduced. In the experiment, the input is set to True.

[0077] Step 4.1.2.1. After processing by the second convolution module, the advanced convolution feature map of the sth blastomere is obtained. ;

[0078] Step 4.1.2.2 The input feature map is processed in the splitting module to obtain the feature map before advanced convolution of the sth blastomere And the feature map after advanced convolution of the sth blastomere .

[0079] Step 4.1.2.3: The convolution module in the Bottleneck_RFMGConv module Perform feature extraction to obtain the deep convolution feature map of the sth blastomere ;

[0080] The multi-dimensional feature fusion module in the Bottleneck_RFMGConv module uses formula (2) to After feature generation processing, the global feature map of the sth blastomere is obtained , then After feature rearrangement processing, the sth blastomere rearrangement feature map is obtained , and use formula (3) to get the sth highly pooled feature , the s-th width pooling feature And the sth global average pooling feature , and then use formula (4) to get the characteristic map of the sth blastomere fusion :

[0081] (2)

[0082] (3)

[0083] (4)

[0084] In formula (2), is the activation function, To normalize the input features, is a two-dimensional convolution operation, is the weight tensor of the convolution kernel in the multi-dimensional feature fusion module, is the stride of the convolution operation;

[0085] In formula (3), is a two-dimensional adaptive average pooling operation, The height dimension of the pooled output size is h, and the width dimension is 1; h represents the height of the input feature map, and w represents the width of the input feature map. Represents the weight tensor of the convolution kernel in the average pooling module; Indicates that the input feature map is at position The value at

[0086] In formula (4), , , Respectively represent the weight tensors in the convolution kernels used in the corresponding height pooling, width pooling, and global average pooling operations; in this embodiment, The value is 2.

[0087] Step 4.1.2.4 After the feature extraction process of the third convolution module, the standard fusion feature map of the sth blastomere is obtained. .

[0088] Step 4.1.3, when p=1, Input into the pth enhanced downsampling feature fusion module, and after being processed by a downsampling module, the pth blastomere convolution downsampling feature map is obtained. After being processed by a C2f_RFMGConv module, the pth blastomere fusion feature map is obtained. ;

[0089] When p=2,3,…,P, fuse the p-1th blastomere with characteristic graphs The input is processed in the p-th enhanced downsampling feature fusion module, so that the P-th enhanced downsampling feature fusion module outputs the final P-th blastomere deep fusion convolution feature map. .

[0090] Step 4.2, the neck network includes in sequence: L enhanced upsampling feature fusion modules, a fourth convolution module, a first C2f_RFMGConv module, and a C2fCIB module; wherein each enhanced upsampling feature fusion module includes: an upsampling module and a C2f_RFMGConv module; in this embodiment, 2 enhanced upsampling feature fusion modules are included;

[0091] Step 4.2.1: hour, Input neck network In the enhanced upsampling feature fusion module, and after being processed by an upsampling module, the first Blastomere upsampling convolution feature map ,Will Enter to The enhanced upsampling feature fusion module is processed in the C2f_RFMGConv module to obtain the Characteristic diagram of deep fusion of blastomeres In this embodiment, the nearest neighbor interpolation method is used for upsampling, and the width magnification factor of the two upsampling modules is 2, and the height remains unchanged.

[0092] when =2,3,…,L, the Characteristic diagram of deep fusion of blastomeres Enter the The Lth enhanced upsampling feature fusion module is used for processing, so that the final Lth blastomere fusion feature map is output by the Lth enhanced upsampling feature fusion module. ;

[0093] Step 4.2.2 Input into the fourth convolution module for feature extraction to obtain the feature map of the i-th blastomere ;

[0094] Step 4.2.3 Input into the first C2f_RFMGConv module for processing to obtain the feature map after the fusion of the i-th blastomere ;

[0095] Step 4.2.3 After being input into the C2fCIB module for processing, the global feature map of the i-th blastomere is obtained. In this embodiment, the number of channels output by the C2fCIB module is 3, and the Boolean value parameter outputs are all set to True.

[0096] Step 4.3, the head network includes: M convolutional layers, H pooling layers, and a prediction head; in this embodiment, it includes 4 convolutional layers and 1 pooling layer;

[0097] Input into the head network, and after feature extraction through M convolutional layers, the convolution feature map of the i-th blastomere is obtained. ;

[0098] After downsampling in H pooling layers, the downsampling feature map of the i-th blastomere is obtained. ;

[0099] Predict Head Pair After processing, we get The predicted coordinates of the j-th blastomere bounding box and the predicted class probabilities of blastomere targets ;in, They are The predicted coordinates of the upper left corner and the upper right corner of the j-th blastomere, , They are The predicted coordinates of the lower left corner and the lower right corner of the j-th blastomere.

[0100] Step 4.4: Construct the total loss of the blastomere target detection network ;

[0101] Step 4.4.1: Use formula (5) to construct the bounding box loss :

[0102] (5)

[0103] In formula (5), J represents The total number of blastomeres predicted in ;

[0104] Step 4.4.2: Use formula (6) to construct the cross entropy loss :

[0105] (6)

[0106] Step 4.4.3: Use formula (7) to construct the total loss :

[0107] (7)

[0108] Step 5: Based on , use the gradient descent method to train the blastomere target detection network and calculate the total loss To update the network parameters until the total loss After convergence or reaching the maximum number of iterations, the training is stopped to generate a blastomere detection model corresponding to the optimal parameters, which is used to detect blastomeres in the test set embryo image sample set.

[0109] The experiment is based on 64-bit operating system Ubuntu 18.04, and the deep learning framework is Pytorch v1.8.0. The CPU is AMD(R) Epyc 7r32 48-Core, and the GPU is NVIDIA GeForce RTX4090. The Adam optimizer is used for training, with the initial learning rate set to 0.01 and the weight decay to 0.0005. If no improvement in the test loss is observed on the validation set for 15 consecutive epochs, early stopping is implemented. The batch size is set to 16.

[0110] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0111] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0112] The experiment uses embryo images collected in Chongqing Maternal and Child Health Hospital as the data set, and the number of various types of cells in the data set is shown in Table 1. True positive (TP), false negative (FN) and recall rate (R) are used as evaluation indicators. True positive (TP) represents the number of correctly detected blastomeres in all blastomeres, false negative (FN) represents the number of blastomeres that have not been detected, also called the number of missed detections, and recall rate (R) represents the proportion of blastomeres that can be correctly detected.

[0113] Table 1 Number of cell images of each type in the embryo dataset

[0114]

[0115] The trained network was tested on the test set and compared with the detection effects of the three networks, Transformer, Yolov5, and Yolov10. It has obvious high detection accuracy in 4-8 cells, as shown in Table 2. The typical blastomere detection display picture is as follows Figure 5 shown.

[0116] Table 2 Comparison of experimental results on cells 1-8

[0117]

[0118] Table 2 and Figure 5 The common results show that the present invention can significantly improve the effect of overlapping blastomere detection. Ours represents the network traffic prediction model proposed by the present invention. Transformer, Yolov5, and Yolov10 represent different network models. Through comparative experiments, it can be seen that the overlapping blastomere detection model proposed by the present invention achieves the best effect in terms of evaluation indicators. The present invention has good application space and prospects, and provides methods and ideas for practical applications in embryo assessment in the future.

Claims

1. A method for detecting overlapping blastomere targets based on multi-dimensional feature fusion, characterized in that: The steps are as follows: Step 1: Obtain the embryo image dataset of the 1-8 cell period and perform preprocessing to obtain the embryo image sample set , represents the i-th embryo image sample, N represents the total number of embryo image samples, The blastomere region divided in the figure is marked to obtain The label, The real position label frame of the jth blastomere is recorded as ,in, , They are The coordinates of the upper left corner and upper right corner of the j-th blastomere, , are the coordinates of the lower left corner and lower right corner of the jth blastomere respectively; Step 2: Perform Fourier transform and get Phase With amplitude , and then use formula (1) to get New phase of : (1) In formula (1), Represents the weight value, Represents the jth embryo image sample The phase of Step 3: New phase of With amplitude Perform inverse Fourier transform to obtain the new i-th embryo image sample , thus forming a new embryo image sample set ; Step 4: Build a blastomere target detection network, including: backbone network, neck network and head network; Step 4.1: The backbone network Processing is performed to obtain the corresponding blastomere deep fusion convolution feature map ; Step 4.2: The neck network Processing is performed to obtain the global feature map of the i-th blastomere ; Step 4.3, the head network includes: M convolutional layers, H pooling layers, and a prediction head; Input into the head network, and after feature extraction through M convolutional layers, the convolution feature map of the i-th blastomere is obtained. ; After downsampling in H pooling layers, the downsampling feature map of the i-th blastomere is obtained. ; Predict Head Pair After processing, we get The predicted coordinates of the j-th blastomere bounding box and the predicted class probabilities of blastomere targets ;in, They are The predicted coordinates of the upper left corner and the upper right corner of the j-th blastomere, , They are The predicted coordinates of the lower left corner and the lower right corner of the j-th blastomere; Step 4.4: Construct the total loss of the blastomere target detection network ; Step 5: Based on , use the gradient descent method to train the blastomere target detection network and calculate the total loss To update the network parameters until the total loss After convergence or reaching the maximum number of iterations, the training is stopped to generate a blastomere detection model corresponding to the optimal parameters, which is used to detect blastomeres in the test set embryo image sample set.

2. A method for detecting overlapping blastomeres based on multidimensional feature fusion according to claim 1, characterized in that: The backbone network in step 4.1 includes, in sequence: a first convolution module, S enhanced convolution feature fusion modules, and P enhanced downsampling feature fusion modules; wherein each enhanced convolution feature fusion module includes: a convolution module and a C2f_RFMGConv module, and each enhanced downsampling feature fusion module includes: a downsampling module and a C2f_RFMGConv module; Step 4.1.1 The first convolution module of the input backbone is used for feature extraction to generate Initial characteristics of blastomere ; Step 4.1.2, when s=1, Input into the sth enhanced convolution feature fusion module and after being processed by a convolution module, we get Standard convolution feature map of the sth blastomere After being processed by a C2f_RFMGConv module, the standard fusion feature map of the sth blastomere is obtained. ; Step 4.1.3, when p=1, Input into the pth enhanced downsampling feature fusion module, and after being processed by a downsampling module, the pth blastomere convolution downsampling feature map is obtained. After being processed by a C2f_RFMGConv module, the pth blastomere fusion feature map is obtained. ; When p=2,3,…,P, fuse the p-1th blastomere with characteristic graphs The input is processed in the p-th enhanced downsampling feature fusion module, so that the P-th enhanced downsampling feature fusion module outputs the final P-th blastomere deep fusion convolution feature map. .

3. A method for detecting overlapping blastomeres based on multidimensional feature fusion according to claim 2, characterized in that: The C2f_RFMGConv module in step 4.1.2 is composed of the second convolution module, the feature map splitting module, the Bottleneck_RFMGConv module, and the third convolution module in sequence; Step 4.1.2.

1. After processing by the second convolution module, the advanced convolution feature map of the sth blastomere is obtained. ; Step 4.1.2.2 The input feature map is processed in the splitting module to obtain the feature map before advanced convolution of the sth blastomere And the feature map after advanced convolution of the sth blastomere ; Step 4.1.2.3: The convolution module in the Bottleneck_RFMGConv module Perform feature extraction to obtain the deep convolution feature map of the sth blastomere ; The multi-dimensional feature fusion module in the Bottleneck_RFMGConv module uses formula (2) to After feature generation processing, the global feature map of the sth blastomere is obtained , then After feature rearrangement processing, the sth blastomere rearrangement feature map is obtained : (2) In formula (2), is the activation function, For normalization, is a two-dimensional convolution operation, is the weight tensor of the convolution kernel in the multi-dimensional feature fusion module, is the stride of the convolution operation; Using formula (3), we can get the s-th highly pooled features: , the s-th width pooling feature And the sth global average pooling feature , and then use formula (4) to get the characteristic map of the sth blastomere fusion : (3) (4) In formula (3), is a two-dimensional adaptive average pooling operation, The height dimension of the pooled output size is h, and the width dimension is 1; h represents the height of the input feature map, and w represents the width of the input feature map. Represents the weight tensor of the convolution kernel in the average pooling module; Indicates that the input feature map is at position The eigenvalue at ; In formula (4), , , Respectively represent the weight tensors in the convolution kernels used in the corresponding height pooling, width pooling, and global average pooling operations; Step 4.1.2.4 After the feature extraction process of the third convolution module, the standard fusion feature map of the sth blastomere is obtained. .

4. A method for detecting overlapping blastomeres based on multi-dimensional feature fusion according to claim 3, characterized in that: The neck network in step 4.2 includes, in sequence: L enhanced upsampling feature fusion modules, a fourth convolution module, a first C2f_RFMGConv module, and a C2fCIB module; wherein each enhanced upsampling feature fusion module includes: an upsampling module and a C2f_RFMGConv module; Step 4.2.1: hour, Input neck network In the enhanced upsampling feature fusion module, and after being processed by an upsampling module, the first Blastomere upsampling convolution feature map ,Will Enter to The enhanced upsampling feature fusion module is processed in the C2f_RFMGConv module to obtain the Characteristic diagram of deep fusion of blastomeres ; when =2,3,…,L, Characteristic diagram of deep fusion of blastomeres Enter the The Lth enhanced upsampling feature fusion module is used for processing, so that the final Lth blastomere fusion feature map is output by the Lth enhanced upsampling feature fusion module. ; Step 4.2.2 Input into the fourth convolution module for feature extraction to obtain the feature map of the i-th blastomere ; Step 4.2.3 Input into the first C2f_RFMGConv module for processing to obtain the feature map after the fusion of the i-th blastomere ; Step 4.2.3 After being input into the C2fCIB module for processing, the global feature map of the i-th blastomere is obtained. .

5. A method for detecting overlapping blastomere targets based on multi-dimensional feature fusion according to claim 4, characterized in that: The step 4.4 comprises: Step 4.4.1: Use formula (5) to construct the bounding box loss : (5) In formula (5), J represents The total number of blastomeres predicted in ; Step 4.4.2: Use formula (6) to construct the cross entropy loss : (6) Step 4.4.3: Use formula (7) to construct the total loss : (7)。 6. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the overlapping blastomere target detection method according to any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the overlapping blastomere target detection method according to any one of claims 1 to 5 are executed.

Images