Ultrasound image identification method, system and equipment for extrauterine pregnancy sac and corpus luteum and medium

Abstract

The invention discloses an extrauterine gestational sac and corpus luteum ultrasonic image recognition method, system and device and a medium, which adopt a one-to-one correspondence scheme and are applied to an extrauterine gestational sac and corpus luteum ultrasonic image recognition task. By adopting a transfer learning mode and combining a non-uniform error function, the problems of small sample size, less training data and unbalanced categories are solved, and the recognition performance of the deep neural network can be better improved, so that higher recognition accuracy is obtained in an extrauterine gestational sac and corpus luteum ultrasonic image recognition task.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image recognition of an ectopic gestational sac and a corpus luteum, and in particular to an ultrasonic image recognition method, system, equipment and medium for an ectopic gestational sac and a corpus luteum. Background Art

[0002] Ectopic pregnancy, also known as extrauterine pregnancy, refers to the implantation of the gestational sac outside the endometrium, accounting for about 2% of all pregnancies. More than 90% of ectopic pregnancies occur in the fallopian tubes, which can be manifested as an adnexal mass on ultrasound. Although the sensitivity and specificity of ultrasound in the evaluation of ectopic pregnancy have gradually increased, in the first evaluation, ultrasound examinations may not be able to reach a definite conclusion in up to 40% of patients. One of the difficulties in identification is to distinguish the nature of the adnexal mass.

[0003] In ultrasound evaluation, the identification of adnexal gestational sac and corpus luteum cyst is one of the key steps. Extrauterine gestational sac often appears as a thick-walled cystic mass or a mixed echo mass, while physiological corpus luteum cyst formed by pregnancy reaction is often difficult to distinguish from extrauterine gestational sac. Previous studies have shown that there is no statistical difference between the two in size and wall thickness. Other differences such as echo and blood flow exist, but they are difficult to qualitatively distinguish and are highly dependent on the experience of ultrasound physicians. Inexperienced and less experienced ultrasound physicians have poor ability to distinguish the two directly based on ultrasound images.

[0004] Deep neural networks, especially convolutional neural networks (CNNs), have achieved remarkable results in natural image analysis and have begun to emerge in the field of medical image processing. In the process of ultrasound image recognition of ectopic pregnancy, the identification of the gestational sac and corpus luteum is one of the key steps. The core concept of deep learning is to imitate the human visual system to describe and express information in a hierarchical manner through multi-level processing. By processing the input image layer by layer, deep neural networks can transform the initial low-level features into high-level feature representations that are more relevant to the target. Since deep neural networks are data-driven, training a convolutional neural network from scratch requires a large amount of labeled data and the parameter adjustment process is complex, which is not applicable to small data sets, especially in the case of an extreme lack of medical images with doctor-labeled information.

[0005] In addition, existing deep neural network-based solutions cannot complete the fine-grained recognition tasks of gestational sac and corpus luteum, for example: (1) The Chinese invention patent application with publication number CN118628461A, “A pelvic floor ultrasound image recognition and analysis system based on deep learning”, automatically recognizes and analyzes pelvic floor ultrasound images by adopting a multimodal deep learning model based on convolutional neural network (CNN) and recurrent neural network (RNN / LSTM); however, its task is limited to the pelvic floor ultrasound image recognition task and cannot complete the fine-grained recognition task of the gestational sac and corpus luteum.

[0006] (2) Chinese invention patent application with publication number CN116172613A, “Method for determining regions of interest in ultrasound images, ultrasound imaging device”, discloses an endometrial image classification scheme based on autoencoder momentum update. However, it is a holistic recognition technology for endometrial images, which is aimed at endometrial image recognition and cannot solve the fine-grained recognition of gestational sacs and corpora lutea in the adnexal area outside the endometrium. Summary of the invention

[0007] The purpose of the present invention is to provide an ectopic gestational sac and corpus luteum ultrasonic image recognition method, system, equipment and medium, which can accurately realize the ectopic gestational sac and corpus luteum ultrasonic image recognition.

[0008] The objective of the present invention is achieved through the following technical solutions: A method for identifying an extrauterine gestational sac and a corpus luteum by ultrasonic imaging, comprising: Construct a training set using labeled ultrasound images; A deep neural network for gestational sac and corpus luteum recognition is constructed, a pre-trained ViT-B model is introduced into the deep neural network, and the deep neural network is trained by transfer learning in combination with a training set; during the training process, the input ultrasound image is divided into multiple image blocks by the input processing layer of the deep neural network and then mapped to the embedding space to obtain the embedding vectors of all image blocks, and the classification tags and the position encoding of the image blocks are added and input into the pre-trained ViT-B model in the deep neural network to obtain the classification features, and then the classifier of the deep neural network outputs the prediction probability by using the classification features; the imbalanced error function is calculated by using the prediction probability combined with the annotation of the corresponding ultrasound image, and the deep neural network is trained by transfer learning; wherein, the ViT-B model is a visual converter model, in which the global dependency relationship between the gestational sac and the corpus luteum, including the morphological features and the relative position relationship, is captured by the attention mechanism in the ViT-B model, and the distinguishing features between the gestational sac and the corpus luteum are increased by linear transformation, and finally the classification features are obtained; The ultrasound image to be identified is input into the trained deep neural network, and the prediction probability is output according to the trained deep neural network.

[0009] An extrauterine gestational sac and corpus luteum ultrasonic image recognition system, comprising: A training set construction unit, used for constructing a training set using annotated ultrasound images; A network construction and training unit is used to construct a deep neural network for gestational sac and corpus luteum identification, wherein a pre-trained ViT-B model is introduced into the deep neural network, and the deep neural network is trained by transfer learning in combination with a training set; during the training process, the input ultrasound image is divided into multiple image blocks by the input processing layer of the deep neural network and then mapped to the embedding space to obtain the embedding vectors of all image blocks, and the classification tags and the position encoding of the image blocks are added and input into the pre-trained ViT-B model in the deep neural network to obtain the classification features, and then the classifier of the deep neural network outputs the prediction probability by using the classification features; the unbalanced error function is calculated by using the prediction probability in combination with the annotation of the corresponding ultrasound image, and the deep neural network is trained by transfer learning; wherein the ViT-B model is a visual converter model, in which the global dependency relationship between the gestational sac and the corpus luteum, including the morphological features and the relative position relationship, is captured by the attention mechanism in the ViT-B model, and the distinguishing features between the gestational sac and the corpus luteum are increased by linear transformation, and finally the classification features are obtained; The ultrasonic image recognition unit is used to input the ultrasonic image to be recognized into the trained deep neural network and output the predicted probability according to the trained deep neural network.

[0010] A processing device, comprising: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0011] A readable storage medium stores a computer program, which implements the above method when the computer program is executed by a processor.

[0012] It can be seen from the technical solution provided by the present invention that in the task of identifying ectopic gestational sac and corpus luteum ultrasound images, a deep neural network based on the pre-trained ViT-B (Vision Transformer-Base) model is introduced, and the transfer learning method is adopted in combination with an uneven error function to solve the problems of small sample size, small training data and imbalanced categories, which can better improve the recognition performance of the deep neural network, thereby obtaining a higher recognition accuracy rate in the task of identifying ectopic gestational sac and corpus luteum ultrasound images. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0014] Figure 1 A flow chart of a method for ultrasonic image recognition of an extrauterine gestational sac and corpus luteum provided in an embodiment of the present invention; Figure 2 A schematic diagram of the marking and processing process of an early pregnancy adnexal mass ultrasound image provided by an embodiment of the present invention; Figure 3 A schematic diagram of the overall framework of a method for ultrasonic image recognition of an ectopic gestational sac and corpus luteum provided by an embodiment of the present invention; Figure 4 A schematic diagram of an extrauterine gestational sac and corpus luteum ultrasonic image recognition system provided by an embodiment of the present invention; Figure 5 A schematic diagram of a processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0016] First, the terms that may be used in this article are explained as follows: The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0017] The term "consisting of..." means excluding any technical feature elements not explicitly listed. If this term is used in a claim, it will make the claim closed, so that it does not contain technical feature elements other than the technical feature elements explicitly listed, except for the conventional impurities related to them. If this term only appears in a clause of a claim, it only limits the elements explicitly listed in the clause, and the elements recorded in other clauses are not excluded from the overall claim.

[0018] The following is a detailed description of an ectopic gestational sac and corpus luteum ultrasonic image recognition method, system, device and medium provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in this field. If no specific conditions are specified in the embodiments of the present invention, the conventional conditions in the field or the conditions recommended by the manufacturer are followed. The instruments used in the embodiments of the present invention, which do not specify the manufacturer, are all conventional products that can be purchased commercially.

[0019] Embodiment 1 The embodiment of the present invention provides a method for ultrasonic image recognition of an extrauterine gestational sac and a corpus luteum to solve the current technical gap. The method can be deployed in a computer or other device to perform the following steps: Figure 1 The following steps are shown: Step 1: Use labeled ultrasound images to construct a training set.

[0020] In an embodiment of the present invention, ultrasound images of early pregnancy adnexal masses may be collected first, and then annotated by an expert, wherein the annotations include: a location area of ​​a gestational sac or a corpus luteum and a category label; the location area of ​​the gestational sac or the corpus luteum is represented by a location coordinate (x1, y1, x2, y2), and the above location coordinates correspond to two location points (x1, y1) and (x2, y2), which are two diagonal location points of a rectangular area, and the rectangular area is the location area where the gestational sac or the corpus luteum is located; the category label indicates the category to which it belongs, and the category label y is equal to 0 or 1. When y=0, it indicates that the category to which it belongs is the corpus luteum, and when y=1, it indicates that the category to which it belongs is the gestational sac.

[0021] Afterwards, for each ultrasound image of the early pregnancy adnexal mass, the ultrasound image of the corresponding position area is cut out according to the position area of ​​the gestational sac or corpus luteum included in the annotation, and scaled to a specified size (for example, 224*224 pixels), and the pixel value is scaled normalized to the interval of -1 to 1 to obtain a normalized ultrasound image, which is then processed using data enhancement and combined with the corresponding category label to form a labeled ultrasound image. Exemplarily, the data enhancement method may include: randomly adjusting the brightness and contrast, and randomly rotating the image by a random angle.

[0022] Based on the above processing, all the labeled ultrasound images are obtained to construct a training set.

[0023] In the embodiment of the present invention, the constructed training set is mainly used for subsequent network training. At the same time, in order to test the performance of the trained network, some labeled ultrasound images are also collected at this stage to construct a test set (the data in the test set does not need to be enhanced). Exemplarily, the training set and the test set contain 765 and 178 labeled ultrasound images, respectively.

[0024] like Figure 2As shown, the process of marking and processing the ultrasound image of the adnexal mass in early pregnancy is demonstrated.

[0025] Step 2: construct a deep neural network for identifying the gestational sac and the corpus luteum, introduce a pre-trained ViT-B model into the deep neural network, and train the deep neural network by transfer learning in combination with the training set.

[0026] In an embodiment of the present invention, during the training process, the input ultrasound image is divided into multiple image blocks by the input processing layer of the deep neural network and then mapped to the embedding space to obtain the embedding vectors of all image blocks, and the classification tags and the position encoding of the image blocks are added and input into the ViT-B model pre-trained in the deep neural network to obtain the classification features, and then the classifier of the deep neural network uses the classification features to output the prediction probability; the prediction probability is combined with the annotation of the corresponding ultrasound image to calculate the imbalance error function, and the deep neural network is trained by transfer learning; wherein, in the ViT-B model, the global dependency relationship between the gestational sac and the corpus luteum, including morphological features and relative positional relationships, is captured through the attention mechanism, and the distinguishing features between the gestational sac and the corpus luteum are increased through linear transformation, and finally the classification features are obtained.

[0027] In the embodiment of the present invention, after the ultrasound images in the training set are input into the deep neural network, they are first divided into multiple fixed-size image patches (Patches) at the input processing layer. For example, the size can be 16*16 pixels. After that, each image patch is flattened and mapped into the embedding space (for example, a 768-dimensional embedding space) to obtain the corresponding embedding vector, which is then concatenated with additional classification tags and the position encoding of the image patch as the input of the ViT-B model.

[0028] The deep neural network constructed in the embodiment of the present invention is a model structure of a deep neural network based on a pre-trained ViT-B (Vision Transformer-Base, visual converter) model, wherein the ViT-B model includes multiple stacked Transformer (transformer neural network) encoder layers, each Transformer encoder layer includes a self-attention mechanism and a feedforward neural network; the self-attention mechanism is a multi-head self-attention mechanism (for example, 12 attention heads), and the output of the multi-head self-attention mechanism enters the feedforward neural network for subsequent processing; the first Transformer encoder input is an embedded vector with a classification mark and an image block position encoding added, and its output is used as the input of the second Transformer encoder layer, and the last Transformer encoder layer outputs the features at the classification mark position, that is, the classification features.

[0029] In addition to the above-mentioned self-attention mechanism and feedforward neural network, each Transformer encoder layer is also equipped with a residual connection and a layer normalization layer at the end to improve the training effect and final performance of the network.

[0030] In an embodiment of the present invention, the classifier may be a linear classifier, which is responsible for performing binary classification and outputting predicted probabilities, for example, outputting a numerical value between 0 and 1 as the predicted probability p (i.e., the probability of belonging to the category of a gestational sac), and the recognition result, i.e., the category to which the ultrasound image belongs, may be determined according to the size of the predicted probability p; for example, a threshold is set, and the recognition result is determined according to the relationship between the threshold and the predicted probability p; illustratively, the threshold may be set to 0.5, and when p<0.5, the ultrasound image is identified as a corpus luteum, and when p≥0.5, the ultrasound image is identified as a gestational sac.

[0031] The ultrasound images in the training set all use the above method to obtain the corresponding prediction probability, and the imbalance error function is calculated in combination with the corresponding annotations, which is expressed as: ; in, The meaning of can be found in the previous description, namely Indicates that the category of the corresponding ultrasound image is corpus luteum (i.e., negative sample), Indicates that the category of the corresponding ultrasound image is a gestational sac (i.e., a positive sample), represents the imbalance error function, is a balancing factor used to balance the importance of positive and negative samples. is a focusing factor, usually a non-negative number, which controls the focus on hard-to-classify samples (i.e., samples in the training data that are easily misclassified by the model). For example, it can be set: .

[0032] In combination with the above-mentioned unbalanced error function, the deep neural network is trained by the back propagation algorithm. The relevant training process can refer to conventional technology, which will not be described in detail in the present invention.

[0033] After training, the performance can be evaluated using the aforementioned test set. Using the same scheme as described above, the ultrasound images in the test set are input into the trained deep neural network to obtain the corresponding prediction probability and then determine the recognition result. Then, the recognition result is compared with the labeled category to determine whether it is correct. Finally, the recognition accuracy can be calculated using the following formula to evaluate the performance of the network: ; The total number of samples in the denominator refers to the number of ultrasound images in the test set (for example, 178 as mentioned above), and the number of correctly predicted samples in the numerator refers to the number of ultrasound images with correct recognition results.

[0034] Based on the above scheme, in actual testing, the overall recognition accuracy of the present invention in the test set reached 89.33%, among which the recall rate of the corpus luteum reached 91.8%, and the accuracy rate of the gestational sac reached 88.03%. It can be seen that the present invention has high recognition performance.

[0035] Step 3: Input the ultrasound image to be identified into the trained deep neural network, and output the prediction probability according to the trained deep neural network.

[0036] The deep neural network obtained through the above training can be used in actual ultrasound image analysis, and the prediction probability can be obtained according to the scheme introduced in the previous article.

[0037] After obtaining the predicted probability, the recognition result can be determined according to the scheme introduced above, and subsequent specific applications can be carried out. The specific application direction involved can be selected by the user according to the actual situation. The present invention does not make specific limitations. For example, it can be used to guide low-level and inexperienced ultrasound physicians to judge the correctness of the recognition results they provide, etc.

[0038] As shown in Figure 3, the overall framework of the method used by the present invention for the task of ultrasonic image recognition of ectopic gestational sac and corpus luteum is presented. First, the input ultrasound image is divided into image blocks of fixed size, such as 16 * 16 pixels, and these image blocks are then linearly flattened into one-dimensional pixel vectors. Each image block is converted into a fixed-dimensional ultrasound image block embedding vector through a linear projection layer, and the dimension is set to 768. At the same time, a special [CLS] marker embedding (as an ectopic gestational sac and corpus luteum ultrasound image recognition marker) is added as the start of the sequence, and position encoding is added to all image blocks to retain the spatial information in the ultrasound image, which is particularly important for accurately identifying the positional relationship between the gestational sac and the corpus luteum.

[0039] Next, these preliminarily processed ultrasound image block embedding vector sequences will pass through N (for example, N = 12) Transformer encoder layers in sequence. Taking a single Transformer encoder layer as an example, it mainly includes two key parts: a multi-head self-attention mechanism (with 12 attention heads) and a feedforward neural network.

[0040] In the multi-head self-attention mechanism, each attention head can establish different levels of association between image blocks in the ultrasound image and other image blocks. For the identification of extrauterine gestational sacs and corpora lutea, this means that the global dependencies such as the morphological characteristics and relative position relationship of the gestational sac and corpora lutea can be captured in all directions. For example, the attention mechanism can focus on the difference between the grayscale values ​​of the two in the ultrasound image, thus providing a key basis for subsequent classification. The normalization (LayerNorm) layer plays a role in stabilizing training in this process. It can normalize the input data to a similar numerical range, making the model more efficient and stable when learning the characteristics of the gestational sac and corpora lutea, avoiding learning difficulties caused by large differences in data scales. The activation function introduces nonlinear factors to the model, allowing the model to fit more complex ultrasound image feature patterns, such as identifying subtle changes in blood flow signals around the gestational sac or subtle differences in echoes inside the corpus luteum. These nonlinear features are crucial for accurately distinguishing the two.

[0041] The feedforward neural network is mainly composed of two linear layers and a normalization layer between them. The first linear layer is responsible for performing a preliminary linear transformation on the features processed by the self-attention mechanism, further integrating feature information of different dimensions, mapping the features of the gestational sac and corpus luteum in the ultrasound image to a new feature space, and highlighting the distinguishing features of the two. The middle normalization layer regularizes the data again to accelerate model convergence and ensure stable learning of the model. The second linear layer converts the integrated features into a form suitable for subsequent classification tasks.

[0042] Finally, the output ectopic gestational sac and corpus luteum ultrasound image features (classification features) corresponding to the [CLS] tag are extracted, and the final classification prediction of the ectopic gestational sac and corpus luteum is performed through a classifier (MLP head, usually a simple linear layer); the MLP here is a multi-layer perceptron. The entire model has approximately 86M parameters and uses direct end-to-end training without the need for manually designed feature extractors. This structure, with its self-attention mechanism, can not only effectively handle long-range dependencies in ultrasound images and accurately identify gestational sacs and corpora lutea, but also maintain the simplicity of the structure and good scalability. During training, a large-scale dataset containing ultrasound images is usually used for pre-training, and then fine-tuning is performed on the specific task of ectopic gestational sac and corpus luteum identification to achieve the best recognition accuracy.

[0043] The method provided by the embodiment of the present invention has many innovations and advantages. Deep neural networks play a core role in processing the ultrasonic image recognition task of the gestational sac and corpus luteum. In particular, the pre-trained ViT-B model is introduced, and the problem of small sample size and scarce training data is dealt with by means of transfer learning. This is because, in the task of extrauterine gestational sac and corpus luteum ultrasonic image recognition, it is quite challenging to obtain a large amount of complete gestational sac and corpus luteum ultrasonic image data, and transfer learning can use the general features learned by the ViT-B model on other large-scale image data to quickly adapt to this task, so that the model can have good initialization parameters even with a small amount of gestational sac and corpus luteum ultrasonic image data, laying the foundation for accurate gestational sac and corpus luteum ultrasonic image recognition.

[0044] At the same time, the problem of class imbalance is also a major obstacle in ultrasound image recognition. The present invention adopts an unbalanced error function to resolve this dilemma. Due to the natural differences in the frequency of occurrence and morphological characteristics of the gestational sac and the corpus luteum in ultrasound images, if conventional training methods are used, the model tends to favor categories with a large number of samples, resulting in a significant decrease in the recognition accuracy of a minority of categories. The unbalanced error function can dynamically adjust the loss weight according to the actual proportion of the two types of samples, ensuring that the model pays equal attention to the gestational sac and the corpus luteum during the learning process, thereby improving the overall recognition performance.

[0045] Going deeper into the internal structure of the model, taking a single Transformer encoder layer as an example, the self-attention mechanism enables the model to transcend the spatial position limitations of the image and focus on the correlation information between different regions when processing ultrasound images. For example, the gestational sac may appear as an irregular shape in an ultrasound image, and the relationship between its edges and surrounding tissues is intricate. The self-attention mechanism can simultaneously focus on the internal echo characteristics of the gestational sac, the edge contour characteristics, and the relative position relationship with the surrounding corpus luteum tissue, etc., integrating these multi-dimensional information to provide a comprehensive basis for subsequent decision-making.

[0046] The normalization layer is responsible for normalizing the data output by the self-attention mechanism to ensure the stability of data distribution. Due to differences in equipment and different acquisition angles, the numerical range of ultrasound imaging data fluctuates greatly, and the normalization layer can unify these uneven data to a similar scale, so that subsequent activation functions and feedforward neural networks can work efficiently in a stable input environment, avoiding problems such as gradient vanishing or gradient explosion caused by differences in data magnitude.

[0047] Activation functions, such as the commonly used ReLU function, introduce nonlinear factors in this task. The determination of ultrasound image features cannot be completed with a simple linear combination. The distinction between the gestational sac and the corpus luteum often involves complex morphological and texture feature comparisons. The ReLU function sets negative values ​​to zero and only retains the nonlinear output of the positive semi-axis, allowing the model to capture more complex high-order relationships between image features, providing strong support for the accurate identification of the two types of targets.

[0048] As the last part of the Transformer encoder layer, the feedforward neural network is responsible for feature transformation and information integration. It further maps the ultrasound image features that have been finely processed in the previous steps to a high-dimensional space for complex feature combination and transformation, mining potential discriminative features, and finally outputting a feature vector for determining whether the image is a gestational sac or a corpus luteum, completing a key step in the entire recognition process and helping the solution provided by the present invention achieve higher recognition performance.

[0049] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0050] Embodiment 2 The present invention also provides an extrauterine gestational sac and corpus luteum ultrasonic image recognition system, which is mainly used to implement the method provided in the above embodiment, such as Figure 4 As shown, the system mainly includes: A training set construction unit, used for constructing a training set using annotated ultrasound images; A network construction and training unit is used to construct a deep neural network for gestational sac and corpus luteum identification, wherein a pre-trained ViT-B model is introduced into the deep neural network, and the deep neural network is trained by transfer learning in combination with a training set; during the training process, the input ultrasound image is divided into multiple image blocks by the input processing layer of the deep neural network and then mapped to the embedding space to obtain the embedding vectors of all image blocks, and the classification tags and the position encoding of the image blocks are added and input into the pre-trained ViT-B model in the deep neural network to obtain the classification features, and then the classifier of the deep neural network outputs the prediction probability by using the classification features; the unbalanced error function is calculated by using the prediction probability in combination with the annotation of the corresponding ultrasound image, and the deep neural network is trained by transfer learning; wherein the ViT-B model is a visual converter model, in which the global dependency relationship between the gestational sac and the corpus luteum, including the morphological features and the relative position relationship, is captured by the attention mechanism in the ViT-B model, and the distinguishing features between the gestational sac and the corpus luteum are increased by linear transformation, and finally the classification features are obtained; The ultrasonic image recognition unit is used to input the ultrasonic image to be recognized into the trained deep neural network and output the predicted probability according to the trained deep neural network.

[0051] In an embodiment of the present invention, the construction of a training set using labeled ultrasound images includes: collecting ultrasound images of early pregnancy adnexal masses and their annotations, wherein the annotations include: the location area of ​​the gestational sac or corpus luteum and a category label; then, for each early pregnancy adnexal mass, according to the location area of ​​the gestational sac or corpus luteum contained in the corresponding annotation, the ultrasound image of the corresponding location area is cut out, scaled to a specified size and then normalized to obtain a normalized ultrasound image, which is then processed in a data enhancement manner and combined with the corresponding category label to form a labeled ultrasound image; and a training set is constructed using all the labeled ultrasound images obtained through the processing.

[0052] In the embodiment of the present invention, the ViT-B model includes multiple stacked Transformer encoder layers, Transformer is a transform neural network, and each Transformer encoder layer includes a self-attention mechanism and a feedforward neural network; the self-attention mechanism is a multi-head self-attention mechanism, and the output of the multi-head self-attention mechanism enters the feedforward neural network for subsequent processing; The input of the first Transformer encoder layer is an embedded vector with the classification mark and the position encoding of the image block added, and its output is used as the input of the second Transformer encoder layer. The last Transformer encoder layer outputs the features at the position of the classification mark, that is, the classification features.

[0053] In the embodiment of the present invention, the calculation of the imbalance error function by using the predicted probability in combination with the annotation of the corresponding ultrasound image includes: The category label y in the annotation is equal to 0 or 1. When y=0, it means that the category of the corresponding ultrasound image is corpus luteum, and when y=1, it means that the category of the corresponding ultrasound image is gestational sac; the predicted probability p is a value between 0 and 1, and the category of the corresponding ultrasound image is determined according to the size of the predicted probability p; The calculation method of the imbalance error function is expressed as: ; in, represents the imbalance error function, is the balance factor, is the focusing factor.

[0054] Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0055] Embodiment 3 The present invention also provides a processing device, such as Figure 5 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the methods provided in the aforementioned embodiments.

[0056] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, the memory, the input device, and the output device are connected via a bus.

[0057] In the embodiment of the present invention, the specific types of the memory, input device and output device are not limited; for example: The input device may be a touch screen, an image acquisition device, a physical button or a mouse, etc.; The output device may be a display terminal; The memory may be a random access memory (RAM) or a non-volatile memory such as a disk storage.

[0058] Embodiment 4 The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.

[0059] In the embodiment of the present invention, the readable storage medium as a computer-readable storage medium can be set in the aforementioned processing device, for example, as a memory in the processing device. In addition, the readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk, etc., which can store program codes.

[0060] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.

Claims

1. A method for identifying an extrauterine gestational sac and corpus luteum by ultrasonic imaging, characterized in that: include: Construct a training set using labeled ultrasound images; A deep neural network for identifying gestational sac and corpus luteum is constructed, a pre-trained ViT-B model is introduced into the deep neural network, and the deep neural network is trained by transfer learning in combination with the training set; During the training process, the input ultrasound image is divided into multiple image blocks through the input processing layer of the deep neural network and then mapped to the embedding space to obtain the embedding vectors of all image blocks, and the classification mark and the position encoding of the image block are added and input into the pre-trained ViT-B model in the deep neural network to obtain the classification features, and then the classifier of the deep neural network uses the classification features to output the prediction probability; the prediction probability is combined with the annotation of the corresponding ultrasound image to calculate the imbalance error function, and the deep neural network is trained by transfer learning; wherein, the ViT-B model is a visual converter model, in which the global dependency relationship between the gestational sac and the corpus luteum, including the morphological features and the relative position relationship, is captured through the attention mechanism in the ViT-B model, and the distinguishing features between the gestational sac and the corpus luteum are increased through linear transformation, and finally the classification features are obtained; The ultrasound image to be identified is input into the trained deep neural network, and the prediction probability is output according to the trained deep neural network.

2. The method for ultrasonic image recognition of an extrauterine gestational sac and corpus luteum according to claim 1, characterized in that: The method of constructing a training set using labeled ultrasound images includes: Collect ultrasound images of early pregnancy adnexal masses and their annotations, where the annotations include: the location area of ​​the gestational sac or corpus luteum and the category label; then, for each early pregnancy adnexal mass, according to the location area of ​​the gestational sac or corpus luteum contained in the corresponding annotation, cut out the ultrasound image of the corresponding location area, scale it to a specified size and then normalize it to obtain a normalized ultrasound image, then process it in a data enhancement manner, and combine it with the corresponding category label to form a labeled ultrasound image; The training set is constructed using all the annotated ultrasound images obtained through processing.

3. The method for ultrasonic image recognition of an extrauterine gestational sac and corpus luteum according to claim 1, characterized in that: The ViT-B model includes multiple stacked Transformer encoder layers, Transformer is a transform neural network, and each Transformer encoder layer includes a self-attention mechanism and a feedforward neural network; the self-attention mechanism is a multi-head self-attention mechanism, and the output of the multi-head self-attention mechanism enters the feedforward neural network for subsequent processing; The input of the first Transformer encoder layer is an embedded vector with the classification mark and the position encoding of the image block added, and its output is used as the input of the second Transformer encoder layer. The last Transformer encoder layer outputs the features at the position of the classification mark, that is, the classification features.

4. The method for ultrasonic image recognition of an extrauterine gestational sac and corpus luteum according to claim 1, characterized in that: The calculation of the imbalance error function by using the predicted probability combined with the annotation of the corresponding ultrasound image includes: The category label y in the annotation is equal to 0 or 1. When y=0, it means that the category of the corresponding ultrasound image is corpus luteum, and when y=1, it means that the category of the corresponding ultrasound image is gestational sac; the predicted probability p is a value between 0 and 1, and the category of the corresponding ultrasound image is determined according to the size of the predicted probability p; The calculation method of the imbalance error function is expressed as: ； in, represents the imbalance error function, is the balance factor, is the focusing factor.

5. An extrauterine gestational sac and corpus luteum ultrasonic image recognition system, characterized in that: include: A training set construction unit, used for constructing a training set using annotated ultrasound images; A network construction and training unit is used to construct a deep neural network for identifying a gestational sac and a corpus luteum, wherein a pre-trained ViT-B model is introduced into the deep neural network, and the deep neural network is trained by transfer learning in combination with a training set; During the training process, the input ultrasound image is divided into multiple image blocks through the input processing layer of the deep neural network and then mapped to the embedding space to obtain the embedding vectors of all image blocks, and the classification mark and the position encoding of the image block are added and input into the pre-trained ViT-B model in the deep neural network to obtain the classification features, and then the classifier of the deep neural network uses the classification features to output the prediction probability; the prediction probability is combined with the annotation of the corresponding ultrasound image to calculate the imbalance error function, and the deep neural network is trained by transfer learning; wherein, the ViT-B model is a visual converter model, in which the global dependency relationship between the gestational sac and the corpus luteum, including the morphological features and the relative position relationship, is captured through the attention mechanism in the ViT-B model, and the distinguishing features between the gestational sac and the corpus luteum are increased through linear transformation, and finally the classification features are obtained; The ultrasonic image recognition unit is used to input the ultrasonic image to be recognized into the trained deep neural network and output the predicted probability according to the trained deep neural network.

6. The ectopic gestational sac and corpus luteum ultrasonic image recognition system according to claim 5, characterized in that: The method of constructing a training set using labeled ultrasound images includes: Collect ultrasound images of early pregnancy adnexal masses and their annotations, where the annotations include: the location area of ​​the gestational sac or corpus luteum and the category label; then, for each early pregnancy adnexal mass, according to the location area of ​​the gestational sac or corpus luteum contained in the corresponding annotation, cut out the ultrasound image of the corresponding location area, scale it to a specified size and then normalize it to obtain a normalized ultrasound image, then process it in a data enhancement manner, and combine it with the corresponding category label to form a labeled ultrasound image; The training set is constructed using all the annotated ultrasound images obtained through processing.

7. The ultrasonic image recognition system for extrauterine gestational sac and corpus luteum according to claim 5, characterized in that: The ViT-B model includes multiple stacked Transformer encoder layers, Transformer is a transform neural network, and each Transformer encoder layer includes a self-attention mechanism and a feedforward neural network; the self-attention mechanism is a multi-head self-attention mechanism, and the output of the multi-head self-attention mechanism enters the feedforward neural network for subsequent processing; The input of the first Transformer encoder layer is an embedded vector with the classification mark and the position encoding of the image block added, and its output is used as the input of the second Transformer encoder layer. The last Transformer encoder layer outputs the features at the position of the classification mark, that is, the classification features.

8. The ultrasonic image recognition system for extrauterine gestational sac and corpus luteum according to claim 5, characterized in that: The calculation of the imbalance error function by using the predicted probability combined with the annotation of the corresponding ultrasound image includes: The category label y in the annotation is equal to 0 or 1. When y=0, it means that the category of the corresponding ultrasound image is corpus luteum, and when y=1, it means that the category of the corresponding ultrasound image is gestational sac; the predicted probability p is a value between 0 and 1, and the category of the corresponding ultrasound image is determined according to the size of the predicted probability p; The calculation method of the imbalance error function is expressed as: ； in, represents the imbalance error function, is the balance factor, is the focusing factor.

9. A processing device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images