A method for automatic placental location identification based on point prompts

Abstract

This application provides a point-cue-based automatic placental location identification method, relating to the field of ultrasound image analysis technology. The method includes: constructing a dataset based on medical image information; training a pre-trained model based on the dataset to obtain a target pre-trained model upon completion of training; using the medical image encoder of the target pre-trained model as the medical image encoder of the placental location identification model, and using the original ultrasound images in the dataset and the corresponding point-cue information as input to the placental location identification model to train the model, resulting in a target placental location identification model upon completion of training; wherein, the target placental location identification model is used for automatic placental location identification from target ultrasound images. This method can improve the accuracy of placental location identification.

Description

Technical Field

[0001] This application relates to the field of ultrasound image analysis technology, specifically to a method for automatic placental location identification based on point prompts. Background Technology

[0002] Placental location is primarily determined by an ultrasound physician performing an abdominal or transvaginal ultrasound examination. The main diagnostic criteria include the placental attachment location, the distance of the placental edge from or beyond the internal cervical os, the thickness of the placenta covering the internal cervical os, and the length of the cervical canal. However, if the equipment in primary care hospitals has low resolution, or if the ultrasound physician lacks experience, missed or misdiagnosed cases may occur. Therefore, current technologies suffer from relatively low accuracy in placental location identification. Summary of the Invention

[0003] In a first aspect, the present invention provides a method for automatic placental location identification based on point cues, comprising: constructing a dataset based on original ultrasound images; wherein the dataset includes the original ultrasound images and masking images identifying the placenta and internal cervical os on the original ultrasound images; training a pre-trained model based on the dataset to obtain a target pre-trained model upon completion of training; wherein the pre-trained model includes a random masking module, a medical image encoder, and a medical image decoder; using the medical image encoder of the target pre-trained model as the medical image encoder of the placental location identification model, and using the original ultrasound images in the dataset and the point cues corresponding to the original ultrasound images as inputs to the placental location identification model to train the placental location identification model, thereby obtaining a target placental location identification model upon completion of training; wherein the placental location identification model includes the medical image encoder, a cue encoder, an image cue feature combiner, and a feature classifier, and the target placental location identification model is used to automatically identify the placental location from a target ultrasound image.

[0004] According to one embodiment of the present invention, training the pre-trained model based on the dataset to obtain a target pre-trained model upon completion of training includes: inputting the original ultrasound image and a masking image corresponding to the original ultrasound image into the random masking module; the random masking module dividing the original ultrasound image and the masking image into multiple blocks and numbering each block; selecting several blocks from the masking image as masking blocks; masking the corresponding numbered blocks in the original ultrasound image based on the masking block numbers; and inputting the masked ultrasound image into the medical... A medical image encoder is used to output a medical image representation vector. This vector is then input into a medical image decoder, which outputs a reconstructed ultrasound image. A first difference loss value is calculated between the reconstructed ultrasound image and the original ultrasound image. This first difference loss value is used to optimize the parameters of the medical image encoder. Upon reaching the training completion threshold, the training process is terminated, and a target pre-trained model with the minimum loss on the validation set is selected. The medical image encoder in the target pre-trained model serves as the medical image encoder for the placental location recognition model.

[0005] According to one embodiment of the present invention, the loss function of the pre-trained model includes:

[0006]

[0007] in, The loss value of the pre-trained model. The pixel values ​​of the reconstructed ultrasound image are denoted by N, where N is the number of samples in each training set of the dataset. j Represents the first in the training set j One sample, It is the first in the original ultrasound image i grayscale value of each pixel. It is the first in the restored ultrasound image i The grayscale value of each pixel.

[0008] According to one embodiment of the present invention, the step of using the medical image encoder of the target pre-trained model as the medical image encoder of the placental location recognition model, and using the original ultrasound images in the dataset and the point cue information corresponding to the original ultrasound images as inputs to the placental location recognition model to train the placental location recognition model, and obtaining the target placental location recognition model after training is completed, includes: inputting the original ultrasound images into the medical image encoder, and having the medical image encoder output medical image representation vectors; wherein, during the training of the placental location recognition model, the model parameters of the medical image encoder remain unchanged; and sequentially inputting the coordinates of each point cue information into the cue encoder to obtain a coordinate vector corresponding to each point cue. The point cue embedding vector corresponding to the information is used; the medical image representation vector and the point cue embedding vector are input to the image cue feature combiner, which fuses the medical image features of the original ultrasound image and the point cue information to output a fused multimodal feature vector; the multimodal feature vector is input to the feature classifier, which outputs a placental location label; a second difference loss value is calculated between the placental location label and the actual placental location label; the second difference loss value is used to optimize the parameters of the feature classifier; when the training completion index is reached, a candidate placental location recognition model corresponding to the training process in the current round is obtained; the training process is repeated multiple times to obtain multiple candidate placental location recognition models.

[0009] According to one embodiment of the present invention, the placental location identification results include normal placenta, low-lying placenta, marginal placenta previa, partial placenta previa, and complete placenta previa, wherein the placental location label is... In the placental location label, only one element is 1, and the rest are 0. The placental location category probability distribution output by the placental location recognition model is as follows: .

[0010] According to one embodiment of the present invention, the loss function of the placental location identification model includes:

[0011]

[0012] in, The loss value for the placental location recognition model is denoted by , where N is the number of samples in each training set of the dataset, and 5 indicates that there are a total of 5 placental location labels. i For sample index variables, j For placental category index variable, For the first i The actual placental location of each sample The placental location label output by the placental location recognition model.

[0013] According to one embodiment of the present invention, the method further includes: after training the placental location recognition model, evaluating the performance of multiple candidate placental location recognition models based on the validation set in the dataset; determining the classification performance of each candidate placental location recognition model based on multiple preset evaluation metrics, and determining the candidate placental location recognition model with the best overall performance as the target placental location recognition model.

[0014] According to one embodiment of the present invention, the evaluation metrics include one or more of precision, recall, F1 score, and AUC-ROC.

[0015] According to one embodiment of the present invention, the method further includes: acquiring a target ultrasound image, setting a plurality of points on the target ultrasound image as point identification information for placental location, preprocessing the target ultrasound image to obtain a preprocessed image, and converting the point coordinates according to the preprocessing result to obtain converted point coordinate information; inputting the preprocessed image and the converted point coordinate information into the target placental location identification model, and having the target placental location identification model output the placental category with the highest probability as the placental location identification result of the target ultrasound image.

[0016] According to one embodiment of the present invention, the point coordinates are: The converted point coordinates are The calculation formula for transforming the coordinates of the points includes:

[0017]

[0018] in, The pixel value of the longer side of the target ultrasound image. The pixel value along the x-axis of the target ultrasound image. The pixel value is the y-axis side pixel value of the target ultrasound image. The pixel value of the preprocessed image is M×M, where M is the fixed side pixel value of the preprocessed target ultrasound image.

[0019] Compared with existing technologies, the advantages of this application are as follows: A target placental location recognition model is obtained through a two-stage training architecture. In the pre-training stage, a self-supervised learning strategy is adopted, using a random occlusion module to selectively occlude key areas in ultrasound images, enabling the pre-trained model's medical image encoder to understand the texture features and spatial distribution patterns of placental tissue. The pre-trained medical image encoder is then transferred to the placental location recognition model. The encoder with fixed parameters retains the general placental features learned in the pre-training stage while avoiding the overfitting risk caused by training from scratch. Based on the trained target placental location recognition model, the placental location is automatically identified in the target ultrasound image, thereby improving the accuracy of placental location recognition. Attached Figure Description

[0020] Figure 1 A schematic diagram illustrating the steps of the point-based automatic placental location identification method provided in this application embodiment.

[0021] Figure 2 This is a schematic diagram illustrating the classification of placenta previa as provided in the embodiments of this application.

[0022] Figure 3 The original ultrasound image provided for the embodiments of this application.

[0023] Figure 4 The masking image provided for the embodiments of this application.

[0024] Figure 5 This is a schematic diagram of the structure of the pre-trained model provided in the embodiments of this application.

[0025] Figure 6 This is a schematic diagram illustrating the division of the original image block, the division of the masked image block, and the random masking of the original image, provided for embodiments of this application.

[0026] Figure 7 This is a schematic diagram of the placental location identification model provided in an embodiment of this application.

[0027] Figure 8 This is a schematic diagram of the orientation of the positional information in the original ultrasound image provided in the embodiments of this application.

[0028] Figure 9 This is a schematic diagram of the modified ResNet-50 structure provided in the embodiments of this application.

[0029] Figure 10 This is a schematic diagram of the structure of the feature classifier provided in the embodiments of this application. Detailed Implementation

[0030] The present application will now be described in further detail with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the subject matter of the present application to the following embodiments. All technologies implemented based on the content of the present application fall within the scope of protection of the present application.

[0031] Unless otherwise specified, the terms "upper," "lower," "left," "right," "center," "inner," "outer," and "side" used in the description of specific embodiments of this application to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings, or the orientation or positional relationship in which the product / equipment / device is usually placed during use. These terms are merely for the purpose of facilitating the description of the solution in this application or simplifying the description in specific embodiments, so as to enable those skilled in the art to quickly understand the solution, and do not indicate or imply that a particular device / component / element must have a specific orientation, or be constructed and operated in a specific positional relationship. Therefore, they should not be construed as limitations on this application.

[0032] In the description of the embodiments of this application, technical terms such as "first" and "second" only distinguish one entity or operation from another, and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary or secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0033] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0034] Please refer to Figure 1 , Figure 1 A schematic diagram illustrating the steps of the point-based automatic placental location identification method provided in this application embodiment. The steps of the point-based automatic placental location identification method may include:

[0035] S1. Construct a dataset based on medical image information.

[0036] S2. Train the pre-trained model based on the dataset to obtain the target pre-trained model after training is complete.

[0037] S3. Use the medical image encoder of the target pre-trained model as the medical image encoder of the placental location recognition model, and use the original ultrasound images and the point prompt information corresponding to the original ultrasound images in the dataset as the input of the placental location recognition model to train the placental location recognition model. After training is completed, the target placental location recognition model is obtained.

[0038] In this application embodiment, the placental location identification results include normal placenta, low-lying placenta, marginal placenta previa, partial placenta previa, and complete placenta previa. Please refer to... Figure 2 , Figure 2 This diagram illustrates the classification of placenta previa as provided in the embodiments of this application. Placenta previa is a significant cause of late pregnancy bleeding and preterm birth, and is closely related to perinatal maternal and fetal complications and mortality. It is also a high-risk factor for placenta accreta. According to the "Guidelines for the Diagnosis and Management of Placenta Previa" published in 2020 by the Obstetrics Group of the Obstetrics and Gynecology Branch of the Chinese Medical Association, placenta previa can be divided into two categories: placenta previa and low-lying placenta. Low-lying placenta refers to placenta attached to the lower segment of the uterus, with the distance between the edge of the placenta and the internal cervical os less than 20 mm. Low-lying placenta can be further divided into low-lying placenta and marginal placenta previa. Placenta previa refers to placenta that completely or partially covers the internal cervical os. Placenta previa can be further divided into partial placenta previa and complete placenta previa.

[0039] In this embodiment, the medical imaging information includes the original ultrasound image, a masked image identifying the placenta and internal cervical os on the original ultrasound image, and the placental location diagnosis result corresponding to the original ultrasound image. See also... Figure 3 and Figure 4 , Figure 3 The original ultrasound images provided in the embodiments of this application. Figure 4 The masking image provided in this application embodiment. The original ultrasound image is an abdominal ultrasound image or a transvaginal ultrasound image of a patient after 28 weeks of pregnancy. In the masking image, the grayscale value of the pixels in the placenta and internal cervical os region outlined by the sonographer is set to 255, and the grayscale value of the remaining pixels is set to 0.

[0040] Specifically, the method for constructing the dataset based on medical image information in step S1 is as follows: Each original ultrasound image is used to construct a sample, and the sample features include the patient's medical image and the corresponding mask image; the medical image and the corresponding mask image are preprocessed into a single-channel grayscale image of a fixed size M×M pixels that conforms to the model input; the label of the sample is the diagnostic result of the placental location in the imaging examination; after the dataset is constructed, it is divided into a training set and a validation set according to a certain ratio. The training set is used for training and optimizing the model parameters, and the validation set is used for model selection.

[0041] The pre-trained model was trained using a constructed dataset. The pre-trained model employed self-supervised learning, and its task was to reconstruct partially obscured ultrasound images of the target region. (See also...) Figure 5 , Figure 5 This is a schematic diagram of the structure of the pre-trained model provided in an embodiment of this application. The pre-trained model includes a random occlusion module, a medical image encoder, and a medical image decoder.

[0042] For example, step S2, which involves training the pre-trained model, may include:

[0043] The original ultrasound image and the corresponding mask image are input into the random masking module. The random masking module divides the original ultrasound image and the mask image into multiple blocks and assigns a number to each block. Several blocks are selected from the mask image as masking blocks, and the corresponding numbered blocks in the original ultrasound image are masked based on the masking block numbers. The masked ultrasound image is input into the medical image encoder, which outputs a medical image representation vector. The medical image representation vector is input into the medical image decoder, which outputs a restored ultrasound image. A first difference loss value is calculated between the restored ultrasound image and the original ultrasound image. The first difference loss value is used to optimize the parameters of the medical image encoder. When the training completion index is reached, the training process is terminated, and the target pre-trained model with the minimum loss on the validation set is selected. The medical image encoder in the target pre-trained model is used as the medical image encoder for the placental location recognition model.

[0044] Specifically, the ultrasound image and its corresponding mask image are first input into a random masking module. This module divides both images into K×K blocks. Then, it randomly selects several blocks from the mask image as masking blocks, and masks the corresponding numbered blocks in the ultrasound image by setting the pixels within these masked blocks to random integer values ​​between 0 and 255. The randomly masked ultrasound image is then used as input to the medical image encoder. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram illustrating the division of the original image block, the division of the masked image block, and the random masking of the original image, provided for embodiments of this application.

[0045] Second, the randomly masked ultrasound images are input into the medical image encoder to learn information related to the placental location in the medical images. The structure of the medical image encoder can be, but is not limited to, residual neural networks, encoders of pre-trained large medical image models, etc. The medical image encoder outputs medical image representation vectors.

[0046] Third, the medical image representation vector is input into the medical image decoder. The structure of the medical image decoder can be, but is not limited to, a deconvolutional neural network, a decoding part of a U-Net structure, or a decoder of a pre-trained large medical image model. The medical image decoder outputs the restored ultrasound image.

[0047] Fourth, the model is trained by minimizing the loss using the stochastic gradient descent algorithm. The model loss function uses the mean squared error loss, and the loss function of the pre-trained model is:

[0048]

[0049] in, The loss value of the pre-trained model. The pixel values ​​of the reconstructed ultrasound image are denoted by N, where N is the number of samples in each training set of the dataset. j Represents the first in the training set j One sample, It is the first in the original ultrasound image i grayscale value of each pixel. It is the first in the restored ultrasound image i The grayscale value of each pixel.

[0050] If the loss on the validation set does not decrease for T consecutive rounds or the model is trained to the specified number of rounds, the model training is complete. The model with the minimum loss on the validation set is selected, and the medical image encoder in the pre-trained model is used as the medical image encoder in the placental location recognition model.

[0051] For example, step S3, training the placental location recognition model, may include:

[0052] The original ultrasound image is input to the medical image encoder, which outputs a medical image representation vector. During the training of the placental location recognition model, the model parameters of the medical image encoder remain unchanged. The coordinates of each point cue are sequentially input to the cue encoder to obtain a point cue embedding vector corresponding to each point cue. The medical image representation vector and the point cue embedding vector are input to the image cue feature combiner, which fuses the medical image features of the original ultrasound image and the point cue information to output a fused multimodal feature vector. The multimodal feature vector is input to the feature classifier, which outputs a placental location label. A second difference loss value is calculated between the placental location label and the actual placental location label. This second difference loss value is used to optimize the parameters of the feature classifier. When the training completion indicator is reached, a candidate placental location recognition model corresponding to the training process in the current round is obtained. The training process is repeated multiple times to obtain multiple candidate placental location recognition models.

[0053] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the placental location recognition model provided in an embodiment of this application. The placental location recognition model includes a medical image encoder, a cue encoder, an image-cue feature combiner, and a feature classifier.

[0054] The placental location identification model includes a medical image encoder, a cue encoder, an image cue feature combiner, and a feature classifier. This target placental location identification model is used to automatically identify the placental location in a target ultrasound image. The input to the automatic placental location identification model includes the patient's ultrasound image and the coordinate information of several cue points provided by the sonographer. The output is the identified placental location category, including: normal placenta, low-lying placenta, marginal placenta previa, partial placenta previa, or complete placenta previa.

[0055] Specifically, the placental location recognition model is trained using a training dataset. First, the ultrasound images from the samples are input into a medical image encoder, which is the pre-trained medical image encoder from step 3. During the training of the automatic placental location recognition model, the encoder's model parameters remain unchanged, and the medical image encoder outputs a medical image representation vector. The point clues are several key points provided by the ultrasound physician for placental location determination. In the training samples, these are several points randomly selected from the target region (pixel grayscale value 255) of the masked image. Please refer to [link / reference]. Figure 8 , Figure 8This is a schematic diagram illustrating the orientation of positional information in the original ultrasound image provided in this embodiment. Each point is identified by its positional information in the ultrasound image. The coordinates of each point are sequentially input into the prompt encoder to obtain a prompt encoder with dimension . E The embedding vector; L A few points can be used to get... L Each point cue is embedded in a vector, and the structure of the cue encoder can be, but is not limited to, a multilayer perceptron, a Transformer encoder, etc.; the mapping formula for each point cue is:

[0056]

[0057] Where, coordinates Here, E represents the marker location in the original ultrasound image, and E is the preset dimension of the embedding vector. to The elements that make up the E-dimensional embedding vector collectively encode the spatial semantic information of the coordinate points.

[0058] Secondly, the medical image representation vector and L The embedding vectors of point cues are input into the image cue feature combiner to fuse medical image features and point cue information, and output the fused multimodal feature vector. The structure of the image cue feature combiner can adopt, but is not limited to, cross-attention mechanism, Transformer-based feature fusion module, etc.

[0059] Finally, the fused multimodal feature vector is input into a feature classifier to identify placental location. The placental location label is... The placental location label contains only one element that is 1, indicating whether the placental location is normal, low-lying, marginal, partial, or complete. All other elements are 0. The placental location recognition model outputs a probability distribution for the placental location category as follows: .

[0060] The feature classifier is implemented using a fully connected feedforward neural network with several hidden layers. The loss function of the placental location recognition model is:

[0061]

[0062] in, The loss value for the placental location recognition model is denoted by , where N is the number of samples in each training set of the dataset, and 5 indicates that there are a total of 5 placental location labels. i For sample index variables, j For placental category index variable, For the first iThe actual placental location of each sample The placental location label output by the placental location recognition model.

[0063] The model is trained by minimizing the value of the loss function using the stochastic gradient descent algorithm. The model is considered trained when the loss on the training set does not decrease for several consecutive rounds or when the training reaches a specified number of rounds. This process is repeated multiple times, and the model obtained from each training is used as a candidate placental location recognition model.

[0064] After the placental location recognition model is trained, the method provided in this application embodiment may further include:

[0065] The performance of multiple candidate placental location recognition models is evaluated based on the validation set in the dataset; the classification performance of each candidate placental location recognition model is determined based on multiple preset evaluation metrics, and the candidate placental location recognition model with the best overall performance is determined as the target placental location recognition model.

[0066] Specifically, the medical image data of the validation set can be input into each candidate placental location recognition model to obtain prediction results. Then, the classification performance of each model can be quantitatively evaluated based on preset evaluation indicators, such as accuracy, recall, F1 score and AUC-ROC (Area Under the Receiver Operating Characteristic Curve). By comparing the performance of each model on the validation set, the model with the best overall performance is selected as the target placental location recognition model.

[0067] Furthermore, the steps for automatically identifying the placental location from ultrasound images of a new patient using a target placental location recognition model may include:

[0068] A target ultrasound image is acquired, and several points are set on the target ultrasound image as point identification information for placental location. The target ultrasound image is preprocessed to obtain a preprocessed image, and the point coordinates are transformed according to the preprocessing result to obtain transformed point coordinate information. The preprocessed image and the transformed point coordinate information are input into the target placental location identification model, and the target placental location identification model outputs the placental category with the highest probability as the placental location identification result of the target ultrasound image.

[0069] Specifically, the ultrasound physician first captures a target ultrasound image of a patient. The physician then clicks on several points on the captured image as indicators of placental location. The target ultrasound image is preprocessed. Based on the preprocessed image, the point coordinates are transformed. For each point, the coordinates are assumed to be... The converted point coordinates are The calculation formula for transforming the coordinates of the points includes:

[0070]

[0071] in, The pixel value of the longer side of the target ultrasound image. The pixel value along the x-axis of the target ultrasound image. The pixel value is the y-axis side pixel value of the target ultrasound image. The pixel value of the preprocessed image is M×M, where M is the fixed side pixel value of the preprocessed target ultrasound image.

[0072] By inputting the preprocessed image and point coordinates into the target placental location recognition model, the target placental location recognition model can automatically output the probability distribution of placental location categories and select the category with the highest probability as the patient's placental location category.

[0073] In the above implementation process, a target placental location recognition model is obtained through a two-stage training architecture. In the pre-training stage, a self-supervised learning strategy is adopted, and a random occlusion module is used to selectively occlude key areas in the ultrasound image, enabling the medical image encoder of the pre-trained model to understand the texture features and spatial distribution patterns of placental tissue. The pre-trained medical image encoder is then transferred to the placental location recognition model. The encoder with fixed parameters retains the general placental features learned in the pre-training stage while avoiding the overfitting risk caused by training from scratch. Based on the trained target placental location recognition model, the placental location is automatically identified in the target ultrasound image, thereby improving the accuracy of placental location recognition.

[0074] The following is a description of the practical application of the point-based automatic placental location identification method provided in the embodiments of this application.

[0075] In practical applications, the patient's medical imaging information is first collected. The medical imaging information consists of ultrasound images after 28 weeks of pregnancy, including ultrasound images, masked images of the target area (placenta and internal cervical os region) manually outlined by the physician, and placental location diagnosis results provided by a professional radiologist, which serve as the gold standard for placental location in this invention. The ultrasound images and masked images are the same size. The ultrasound images are grayscale images, and the masked images are binary grayscale images. The grayscale value of pixels inside the target area is 255, and the grayscale value of pixels outside the target area is 0.

[0076] The dataset for training the placental location recognition model is constructed. Each ultrasound image is used to construct a sample. The sample features include the ultrasound image and the target region mask image. The ultrasound image and the mask image need to be preprocessed to a fixed size that conforms to the model input, such as 864×864 pixels. Table 1 is an illustration of how to construct a training sample using the information of an ultrasound image as an example.

[0077] Table 1

[0078]

[0079] First, the patient's ultrasound images are preprocessed. The pixel size is adjusted to 864×864, specifically by stretching the original image proportionally until the longer side is 864 pixels, and then performing symmetrical zero-value filling on the shorter side. Next, grayscale normalization is performed, dividing the grayscale value of each pixel by 255 to normalize the grayscale values ​​of all pixels to the range [0, 1]. The same method is used to preprocess the mask image, adjusting its pixel size to 864×864. The labels for the training samples are then calculated, with the label being the placental location label. According to the medical imaging report, the patient has complete placenta previa; therefore, the placental location label is as follows. The above method is used to create samples, thereby obtaining the dataset.

[0080] After the dataset is constructed, the training set and validation set are divided. The specific method is as follows: on the premise of ensuring the balanced distribution of data, a stratified sampling strategy is used to divide the dataset into a training set and a validation set in a ratio of 4:1. The training set is used for training and optimizing model parameters, and the validation set is used for model selection.

[0081] The medical image encoder was pre-trained using the training set. First, the ultrasound images in the training samples were randomly masked. The specific method is as follows: the ultrasound image and the mask image were divided into 432×432 blocks, each block being 2×2 in size. Five non-overlapping blocks were randomly selected from the target region of the mask image. The gray values ​​of the corresponding pixels in the ultrasound image were set to random values ​​between 0 and 1.

[0082] Second, the randomly masked ultrasound images are input into a processing unit to extract medical image representation vectors. In this implementation, the medical image encoder uses a modified residual neural network, such as ResNet-50. Please refer to [link to relevant documentation]. Figure 9 , Figure 9 This is a schematic diagram of the modified ResNet-50 structure provided in the embodiments of this application. The medical image encoder includes five stages: the first stage contains one initial convolutional layer and max pooling, and the following four stages contain 3, 4, 6, and 3 residual blocks, respectively. This encoder maps the ultrasound image into a three-dimensional vector of size 2048×27×27, where the number of channels is 2048, and the feature matrix size of each channel is 27×27.

[0083] Third, the medical image representation vector is input into the medical image decoder to reconstruct the image. In this specific implementation method, the decoder used is a deconvolutional neural network, which contains 5 deconvolutional layers with a kernel size of 4×4 and a stride of 2. Each deconvolutional layer halves the number of channels and doubles the size. Each deconvolutional layer is followed by a batch normalization layer and a ReLU activation function layer. The last two layers of the decoder are a convolutional layer to reduce the number of channels from 64 to 1 and a Sigmoid activation function layer to map the output value to the [0, 1] interval. Finally, the medical image decoder outputs an 864×864 two-dimensional grayscale matrix.

[0084] During pre-training, the loss value is calculated based on the loss function of the pre-trained model for each training set sample. In this implementation, a stochastic gradient descent optimizer is used to calculate the gradient of the loss with respect to the model parameters through backpropagation, thereby updating the network parameters. Model training is complete when the loss on the validation set does not decrease for 20 consecutive epochs or when the specified number of epochs has been reached. The encoder of the model with the minimum loss on the validation set is used as the medical image encoder for subsequent end-to-end training.

[0085] A medical image encoder with prompts was used to perform end-to-end training of an automatic placental location recognition model, employing a training set and a target pre-trained model. First, prompts were obtained from masked images in the training samples. Five points, each with a value of 255, were randomly selected from the masked images, and their coordinates were concatenated into a 5×2 vector, which served as the prompt vector during training. Then, ultrasound images from the training samples were input into the medical image encoder to extract the medical image representation vector. The parameters of the medical image encoder were frozen during training and remained unchanged.

[0086] Simultaneously, the cue vector is input into the cue encoder to encode placental cue information. In this embodiment, the cue encoder includes a multilayer perceptron, which comprises an input layer, a hidden layer, and an output layer. The input layer has 2 input neurons, the hidden layer contains 128 neurons using the ReLU activation function, and the output layer contains 2048 output neurons. The cue encoder inputs each 2-bit vector from the input cue vector into the multilayer perceptron and averages the five resulting 2048-bit vectors to obtain a one-dimensional cue representation vector of size 2048, which serves as the classification cue feature.

[0087] The cue representation vector output by the image encoder and the medical image representation vector output by the cue encoder are input into a feature combiner to fuse medical image features and classification cue features. In this embodiment, the feature combiner structure is as follows: First, the medical image representation vector is input into a global average pooling layer, converting it into a one-dimensional vector of size 2048. Then, this processed medical image representation vector and the cue representation vector are concatenated into a one-dimensional vector of size 4096, which is the fused feature representation vector.

[0088] Finally, the fused feature representation vector is input into the feature classifier. In this embodiment, the feature classifier is implemented using a multilayer perceptron. Please refer to [link / reference needed]. Figure 10 , Figure 10 This is a schematic diagram of the feature classifier provided in the embodiments of this application, wherein CONV is a convolutional layer, BN (BatchNorm) is batch normalization, BTNK1 is a residual block with downsampling, and BTNK2 is a residual block without downsampling.

[0089] The feature classifier consists of an input layer, a hidden layer, and an output layer. The input layer contains 4096 neurons, the hidden layer contains 512 neurons, and each neuron uses the ReLU function as the activation function. The output layer contains 5 output neurons, and each output neuron uses the Softmax function as the activation function. The probabilities of the patient having a normal placenta, low-lying placenta, marginal placenta previa, partial placenta previa, and complete placenta previa are calculated respectively.

[0090] After training, the performance of multiple candidate models is evaluated using a validation set. This evaluation includes the following steps: First, medical image data from the validation set is input into each candidate model to obtain prediction results. Then, the classification performance of each model is quantitatively evaluated based on pre-defined evaluation metrics (such as loss, accuracy, recall, F1 score, and AUC-ROC). In this specific implementation method, accuracy is used as the primary evaluation metric, and loss as a secondary evaluation metric. By comparing the performance of each model on the validation set, the model with the best overall performance is selected as the final deployment model.

[0091] For a new patient, an ultrasound image is first collected. The ultrasound image is preprocessed to 864×864 pixels and standardized. The standardized medical image is then input into a trained target placental location recognition model. To improve accuracy, the physician can select several points within the placental area as prompts and input them into the model. Finally, the target placental location recognition model outputs the probability of the patient belonging to one of five placental locations, assuming a probability of [0.5, 0.2, 0.1, 0.1, 0.1]. The category with the highest probability is selected as the patient's placental location. In this embodiment, based on the output of the target placental location recognition model, the patient's placental location can be identified as a normal placenta.

[0092] Based on the same concept, embodiments of this application also provide a computer device, which may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described above.

[0093] Based on the same concept, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0094] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for automatic placental location identification based on point prompts, characterized in that, include: A dataset is constructed based on medical imaging information; wherein, the medical imaging information includes original ultrasound images and masked images that identify the placenta and internal cervical os on the original ultrasound images, as well as the placental location diagnosis results corresponding to the original ultrasound images. The pre-trained model is trained based on the dataset to obtain the target pre-trained model upon completion of training; wherein the pre-trained model includes a random occlusion module, a medical image encoder, and a medical image decoder. The medical image encoder of the target pre-trained model is used as the medical image encoder of the placental location recognition model, and the original ultrasound images in the dataset and the point cue information corresponding to the original ultrasound images are used as the input of the placental location recognition model to train the placental location recognition model. When the training is completed, the target placental location recognition model is obtained. The placental location recognition model includes the medical image encoder, the cue encoder, the image cue feature combiner, and the feature classifier. The target placental location recognition model is used to automatically identify the placental location of the target ultrasound image.

2. The method for automatic placental location identification based on point prompts according to claim 1, characterized in that, The step of training the pre-trained model based on the dataset to obtain the target pre-trained model upon completion of training includes: The original ultrasound image and the mask image corresponding to the original ultrasound image are input into the random masking module. The random masking module divides the original ultrasound image and the mask image into multiple blocks and assigns a number to each block. Several blocks are selected from the masked image as masking blocks, and the corresponding numbered blocks in the original ultrasound image are masked based on the number of the masking blocks. The masked ultrasound image is input to the medical image encoder, and the medical image encoder outputs a medical image representation vector. The medical image representation vector is input into the medical image decoder, and the medical image decoder outputs the restored ultrasound image; Calculate a first difference loss value between the restored ultrasound image and the original ultrasound image; wherein, the first difference loss value is used to optimize the parameters of the medical image encoder; Upon reaching the training completion criterion, the training process is terminated and the target pre-trained model with the minimum loss on the validation set is selected; the medical image encoder in the target pre-trained model is used as the medical image encoder for the placental location recognition model.

3. The method for automatic placental location identification based on point prompts according to claim 2, characterized in that, The loss function of the pre-trained model includes: in, The loss value of the pre-trained model. The pixel values ​​of the reconstructed ultrasound image are denoted by N, where N is the number of samples in each training set of the dataset. j Represents the first in the training set j One sample, It is the first in the original ultrasound image i grayscale value of each pixel. It is the first in the restored ultrasound image i The grayscale value of each pixel.

4. The method according to claim 1, characterized in that, The step involves using the medical image encoder of the target pre-trained model as the medical image encoder of the placental location recognition model, and using the original ultrasound images in the dataset and the corresponding point cue information as input to the placental location recognition model to train the placental location recognition model. Upon completion of training, a target placental location recognition model is obtained, comprising: The original ultrasound image is input to the medical image encoder, which outputs a medical image representation vector; wherein, during the training of the placental location recognition model, the model parameters of the medical image encoder remain unchanged. The coordinates of each point prompt information are sequentially input into the prompt encoder to obtain the point prompt embedding vector corresponding to each point prompt information; The medical image representation vector and the point cue embedding vector are input into the image cue feature combiner, which fuses the medical image features of the original ultrasound image and the point cue information to output the fused multimodal feature vector. The multimodal feature vector is input into the feature classifier, and the feature classifier outputs the placental location label. Calculate a second difference loss value between the placental location label and the actual placental location label; wherein, the second difference loss value is used to optimize the parameters of the feature classifier; If the training completion criteria are met, the candidate placental location recognition model corresponding to the training process in the current round is obtained. The training process was repeated multiple times to obtain multiple candidate placental location recognition models.

5. The method for automatic placental location identification based on point prompts according to claim 4, characterized in that, The placental location identification results include normal placenta, low-lying placenta, marginal placenta previa, partial placenta previa, and complete placenta previa. The placental location label is... In the placental location label, only one element is 1, and the rest are 0. The placental location category probability distribution output by the placental location recognition model is as follows: .

6. The method according to claim 5, characterized in that, The loss function of the placental location identification model includes: in, The loss value for the placental location recognition model is denoted by , where N is the number of samples in each training set of the dataset, and 5 indicates that there are a total of 5 placental location labels. i For sample index variables, j For placental category index variable, For the first i The actual placental location of each sample The placental location label output by the placental location recognition model.

7. The method for automatic placental location identification based on point prompts according to claim 4, characterized in that, The method further includes: After the placental location recognition model is trained, the performance of multiple candidate placental location recognition models is evaluated based on the validation set in the dataset. The classification performance of each candidate placental location identification model is determined based on multiple preset evaluation indicators, and the candidate placental location identification model with the best overall performance is determined as the target placental location identification model.

8. The method for automatic placental location identification based on point prompts according to claim 7, characterized in that, The evaluation metrics include one or more of the following: accuracy, recall, F1 score, and area under the receiver operating characteristic curve.

9. The method for automatic placental location identification based on point prompts according to claim 1, characterized in that, The method further includes: Acquire a target ultrasound image and set several points on the target ultrasound image as point identification information for the placental location; The target ultrasound image is preprocessed to obtain a preprocessed image, and the point coordinates are transformed according to the preprocessing result to obtain the transformed point coordinate information. The preprocessed image and the converted point coordinate information are input into the target placental location recognition model, and the target placental location recognition model outputs the placental category with the highest probability as the placental location recognition result of the target ultrasound image.

10. The method for automatic placental location identification based on point prompts according to claim 9, characterized in that, in, The coordinates of the point are The converted point coordinates are The calculation formula for transforming the coordinates of the points includes: in, The pixel value of the longer side of the target ultrasound image. The pixel value along the x-axis of the target ultrasound image. The pixel value is the y-axis side pixel value of the target ultrasound image. The pixel value of the preprocessed image is M×M, where M is the fixed side pixel value of the preprocessed target ultrasound image.

Images