Ultrasound image segmentation system training method and nerve block auxiliary system

Abstract

The invention discloses an ultrasonic image segmentation system training method and a nerve block auxiliary system, and belongs to the field of medical image segmentation, and the method comprises the steps: proposing a semi-supervised segmentation framework of progressive learning from simple to difficult, and explicitly generating a simplified sample and a complicated sample through the bidirectional replacement of a difficult region and a simple region, and a gradient weight mechanism is introduced, and the contribution strength of the two types of samples in each training stage is dynamically regulated and controlled. The nerve block auxiliary system comprises an ultrasonic device, a data acquisition device, a processing system and an interaction device, an ultrasonic image segmentation system trained by the provided method is deployed in the processing system, and the processing system is used for inputting a to-be-segmented ultrasonic image into the ultrasonic image segmentation system so as to carry out segmentation and visualization processing on the ultrasonic image. And the segmentation result and the visualization result are visually displayed through the interaction device. The segmentation performance of the ultrasonic image segmentation system can be improved, so that nerve block is effectively assisted.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation, and more specifically, relates to a training method for an ultrasound image segmentation system and a nerve block assist system. Background Technology

[0002] Nerve block is a highly effective method of local anesthesia and analgesia, achieved by injecting local anesthetics around the target nerve or nerve plexus. Ultrasound imaging, as a safe, real-time, and low-cost imaging method, has become one of the most widely used examination tools in clinical practice. Currently, hospitals often perform nerve blocks by observing real-time ultrasound images, i.e., ultrasound-guided nerve blocks.

[0003] Manual screening cannot keep up with the rapid increase in the number of ultrasound images. With the development of deep learning technology, using deep learning to automatically segment ultrasound images has become the answer to the above problems. Currently, a large number of excellent ultrasound image segmentation models have been proposed. However, traditional fully supervised methods are highly dependent on the quantity and quality of labeled data. Obtaining labels is an expensive and tedious task, especially for ultrasound images. First, the labeling of ultrasound images is different from the common-sense labeling of natural images, such as the labeling of tall buildings, vehicles, sky, and roads in street images, which can be labeled with only common-sense knowledge. The labeling of ultrasound images requires the labeler to have a lot of clinical knowledge, and often requires clinicians to do the labeling. Second, the inherent characteristics of ultrasound images, such as low signal-to-noise ratio, blurred boundaries, and artifact interference, bring severe challenges to labeling. These reasons make it much more difficult to obtain ultrasound image labels than natural images and other medical images.

[0004] To address the difficulty of obtaining labeled data, an increasing number of researchers are shifting their focus to semi-supervised image segmentation. Semi-supervised image segmentation aims to train high-performance models using a small number of labeled images and a large number of unlabeled images. Early semi-supervised image segmentation methods attempted to incorporate Generative Adversarial Networks (GANs), using the trained segmentation model as a generator to produce predictions that could deceive the discriminator and thus utilize unlabeled data. However, due to the exceptionally difficult training process, this method is rarely used now. Currently, the mainstream semi-supervised image segmentation methods are mainly divided into pseudo-label methods and consistency regularization methods.

[0005] The common approach to pseudo-labeling is to first train a teacher model using labeled data, then use the teacher model to predict pseudo-labels for unlabeled images, and finally retrain a student model using these pseudo-labels. Although pseudo-labeling is effective, its three-stage operation makes it too cumbersome in practice. Recent research tends to use consistency regularization. The core idea of ​​consistency regularization is to ensure that the model's predictions of the same image under different perturbations remain consistent. By minimizing this difference, the model is optimized using unlabeled images. Specifically, the model's predictions of clean images (images with simple perturbations such as cropping, scaling, and flipping) are used to supervise the model's predictions of strongly enhanced images (images with difficult perturbations such as color transformation, Gaussian blur, and CutMix).

[0006] Existing semi-supervised image segmentation methods can solve the problem of difficulty in obtaining annotations to some extent. However, in existing semi-supervised image segmentation methods, the segmentation model cannot effectively utilize the information of labeled and unlabeled images during the training process. The segmentation performance of the final trained segmentation model still needs to be further improved. This makes ultrasound-guided nerve block surgery relatively difficult to perform. Summary of the Invention

[0007] To address the shortcomings and improvement needs of existing technologies, this invention provides a training method for an ultrasound image segmentation system and a nerve block assist system, with the aim of improving the segmentation performance of the ultrasound image segmentation system and thus effectively assisting in nerve block.

[0008] To achieve the above objectives, according to one aspect of the present invention, a training method for an ultrasound image segmentation system is provided, comprising: training a teacher network and a student network for image segmentation using labeled and unlabeled images; after training, using the student network as the ultrasound image segmentation system; and in each training step of each training round, the training includes the following steps: The labeled image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the segmentation label of the labeled image is calculated. ; Unlabeled images are input into the teacher network to obtain segmentation masks and prediction confidence. The segmentation masks are used as pseudo-labels for the unlabeled images. For each unlabeled image, it is divided into multiple image blocks. The average entropy of each image block is calculated based on the prediction confidence. The image blocks with the largest and smallest average entropy are identified as difficult and easy regions, respectively. Divide the unlabeled image into multiple image pairs; for each image pair... ,use Simple region replacement In difficult regions, simple, strongly enhanced images are obtained, and the following methods are utilized: The part corresponding to the simple area in the pseudo tag, covering The pseudo-labels are obtained by identifying the positions of the original difficult regions in the pseudo-labels; using... Difficult area replacement From simple regions, difficult-to-enhance images are obtained, and the following methods are utilized: The part of the pseudo-tag corresponding to the difficult area is covered. The pseudo-labels correspond to the original simple regions in the pseudo-labels, and the pseudo-labels of the enhanced image are obtained. and ; A simple strongly augmented image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the pseudo-label is calculated. The hard-enhanced image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the pseudo-label is calculated. ; according to Calculate the loss of the current training step And based on the loss Update the parameters for the student network, and then update the parameters for the teacher network; Among them, the labeled image is the ultrasound image with the target region mask labeled, and the unlabeled image is the ultrasound image without the target region mask labeled; and Losses and The weight.

[0009] Furthermore,

[0010]

[0011] in, Indicates the current training round number. This indicates the total number of training rounds.

[0012] Furthermore, after identifying the image patches with the largest and smallest average entropy as difficult and simple regions respectively, the process also includes: randomly extending the difficult and simple regions, and using the extended difficult and simple regions as the final difficult and simple regions. Randomly extend the difficult and easy regions, including: The image patch with the largest average entropy is identified as the center of the difficult region. A random-sized extension is then made outward from the center of the difficult region on the unlabeled image, and the extended region is taken as the final difficult region. The image patch with the smallest average entropy is identified as the center of a simple region. A random-sized extension is then made outward from the center of the simple region on the unlabeled image, and the extended region is taken as the final simple region. In this case, after extension, the difficult region is the same size as the easy region; the size of the extended difficult region... satisfy , The size of the difficult area before extension. The size of the unlabeled image. This is a preset ratio.

[0013] Furthermore, the first Average entropy of image patches The calculation formula is as follows:

[0014] in, Indicates the total number of categories. and These represent the height and width of the image patch, respectively. Indicates the first The first image patch Each pixel belongs to the category The confidence level.

[0015] Further, calculate the loss. Previously, it also included: The simple strongly enhanced image is input into the teacher network to obtain a segmentation mask and prediction confidence. The segmentation mask contains prediction confidence values ​​less than a preset threshold. Pixel removal; Furthermore, calculate the loss. Previously, it also included: The hard-enhanced image is input into the teacher network to obtain a segmentation mask and prediction confidence. Within this segmentation mask, images with prediction confidence less than a threshold are selected. Pixel removal.

[0016] Furthermore, based on the loss Update the parameters for the student network, and then update the parameters for the teacher network, including: According to the loss Calculate the gradient and update the parameters of the student network using the gradient descent algorithm. ; according to Update the parameters of the teacher network; in, To update the teacher network parameters, To update the previous teacher network parameters; The preset weighting coefficients, and , This indicates the number of training steps.

[0017] Furthermore, the student network uses DINOV2-S as the encoder.

[0018] According to another aspect of the present invention, a nerve block assistive system is provided, comprising: an ultrasound device, a data acquisition device, a processing system, and an interactive device; Ultrasound equipment is used to acquire ultrasound images of patients during nerve blocks; One end of the data acquisition device is connected to the ultrasound equipment, and the other end is connected to the processing system. It is used to transmit the ultrasound images acquired by the ultrasound equipment to the processing system. The processing system is equipped with an ultrasound image segmentation system, which is trained by the ultrasound image segmentation system training method provided by the present invention. One end of the processing system is connected to a data acquisition device, and the other end is connected to an interactive device. The system is used to input the ultrasound image to be segmented into the ultrasound image segmentation system to segment the ultrasound image and transmit the segmentation result to the interactive device. An interactive device, connected to the processing system, is used to display the received segmentation results and visualization results.

[0019] Furthermore, the processing system is also used to visualize the ultrasound image based on the segmentation mask obtained from the segmentation, and transmit the visualization results to the interactive device.

[0020] Furthermore, the visualization process includes: The regions where each anatomical structure is located are obtained based on the segmentation mask. Different colors are then used to cover the regions where different anatomical structures are located on the original patient ultrasound image to obtain the display image.

[0021] Furthermore, the visualization process also includes: determining whether the segmentation result meets the preset conditions; if it does, the original ultrasound image is determined to be a standard image; otherwise, the original ultrasound image is determined to be a non-standard image; and calculating the degree of conformity between the original ultrasound image and the reference standard image. The reference standard image is an ultrasound image that provides the best puncture indication for nerve block.

[0022] Furthermore, the preset conditions include: the appearance of the target anatomical structure, and at least one of the following conditions: The outline of the target anatomical structure is clear; the position of the target anatomical structure is relatively stable; the size of the target anatomical structure is greater than the preset first threshold; the degree of conformity between the original ultrasound image and the reference standard image is greater than the preset second threshold.

[0023] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: (1) In this invention, we consider that in image segmentation, the boundary of the segmentation target is relatively more difficult to learn and the corresponding prediction confidence is low, while the center of the segmentation target and the background region are easier to learn and the prediction confidence is high. Therefore, we divide the difficult region and the simple region in each unlabeled ultrasound image and generate consistency regularization by complementing the difficult and simple regions between unlabeled images. Specifically, we pair the unlabeled images in pairs. In each image pair, we use the simple region in one image to cover the difficult region in the other image to generate a simple image containing more background and the center region of the segmentation target. We also use the difficult region in one image to cover the simple region in the other image to generate a difficult image containing more boundary regions of the segmentation target. The former enables the model to quickly learn a more stable segmentation result, while the latter can optimize the model's ability to segment regions that are not yet mastered. This gives full play to the role of the two types of images and effectively improves the segmentation performance of the trained ultrasound image segmentation system, thereby effectively assisting neural blockade.

[0024] (2) This invention considers that the model should emphasize regions with different learning difficulties at different training stages. The model should first focus on training simple regions and then strengthen the learning of difficult regions. Therefore, in the preferred embodiment of this invention, a progressive complementarity between easy and difficult regions is achieved through weight settings. Specifically, according to... ( (Indicates the current training round number) The weights of the segmentation loss for difficult strongly augmented images and simple strongly augmented images are set. In the early stages of training, simple strongly augmented images, which contain more background and the center of the segmented target, are given a larger weight, allowing the model to quickly learn stable segmentation results. As training progresses, the weight of simple strongly augmented images gradually decreases, while the weight of difficult strongly augmented images, which contain more boundary regions of the segmented target, gradually increases. As a result, the training focus of the model gradually shifts from learning stable segmentation results to improving the segmentation performance of poorly segmented regions, thereby further improving the segmentation performance.

[0025] (3) Based on the training method of the ultrasound image segmentation system proposed in this invention, the ultrasound image segmentation system deployed in the nerve block auxiliary system provided by this invention can identify key structures in the image more quickly and accurately, providing effective guidance information for nerve block, thereby effectively assisting nerve block; on this basis, this invention uses color to attach different anatomical structures, and based on the recognition results, it can more quickly determine whether the size, position and other information of each anatomical structure conforms to the standard image. By attaching color to key structures in the image and providing the degree of conformity between the original ultrasound image and the standard image, the difficulty of implementing nerve block can be significantly reduced. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the training method for the ultrasound image segmentation system provided in an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the structure of DINOV2-S provided in an embodiment of the present invention.

[0028] Figure 3 A schematic diagram of the nerve block assist system provided by the present invention.

[0029] Figure 4 The flowchart of the processing system segmenting ultrasound images in the nerve block assist system provided by the present invention.

[0030] Figure 5 This is a schematic diagram of the ultrasound image visualization results processed by the processing system in the nerve block assist system provided by the present invention.

[0031] Figure 6 The overall workflow diagram of the nerve block assist system provided by the present invention.

[0032] Figure 7 The specific workflow diagram of the nerve block assist system provided by the present invention is shown below.

[0033] Figure 8 Visual segmentation results of different methods provided by this invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0035] In this invention, the terms "first," "second," etc. (if present) in the invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0036] To improve the training method of ultrasound image segmentation systems in semi-supervised scenarios by incorporating the characteristics of ultrasound images, thereby enhancing the segmentation performance of ultrasound image segmentation systems, this invention provides a training method for ultrasound image segmentation systems. Compared to adding various other modules, this invention focuses on the ultrasound image itself, arguing that different regions in the ultrasound image play different roles in model training. Simple regions (background, center of the segmentation target) enable the model to quickly learn more stable segmentation results, while difficult regions (boundaries of the segmentation target) allow the model to further improve upon its stable segmentation of simple regions. To this end, this invention proposes a data augmentation method that complements easy and difficult regions. Through bidirectional replacement of simple and difficult regions, a Simple Strong Augmentation Image (ESAI) containing more simple regions and a Hard Strong Augmentation Image (HSAI) containing more difficult regions are obtained. By fully utilizing the roles of different regions in the image through ESAI and HSAI, the segmentation performance of the trained ultrasound image segmentation system is effectively improved.

[0037] Based on the bidirectional complementarity of easy and difficult regions, this invention further proposes that the model should emphasize different regions of difficulty at different training stages. The easy regions should be trained first, and then the difficult regions should be strengthened. This learning process is in line with the idea of ​​curriculum learning. To this end, this invention designs progressive loss weights to further strengthen the role of ESAI and HSAI at different training stages of the model.

[0038] In terms of model structure, this invention introduces the cutting-edge DINOV2 encoder into the field of ultrasound image segmentation, replacing the outdated ResNet encoder and replacing the ResNet34 and ResNet50 encoders commonly used in ultrasound image segmentation, thereby achieving higher ultrasound image segmentation performance.

[0039] The following is an example.

[0040] Example 1: A training method for an ultrasound image segmentation system includes: training a teacher network and a student network for image segmentation using labeled and unlabeled images; after training, using the student network as the ultrasound image segmentation system.

[0041] In this embodiment, the labeled image is an ultrasound image with a target region mask labeled, and the unlabeled image is an ultrasound image without a target region mask labeled.

[0042] like Figure 1 As shown, in this embodiment, in each training step of each training round, both labeled and unlabeled images are used simultaneously to obtain a labeled image batch: Unlabeled image batch: The training in each training step of each training round includes the following steps S1~S5: S1: Input the labeled image into the student network to obtain the segmentation mask, and calculate the loss between the segmentation mask and the segmentation label of the labeled image. ; S2: Input the unlabeled image into the teacher network to obtain the segmentation mask and prediction confidence. Use the segmentation mask as the pseudo-label of the unlabeled image. For each unlabeled image, divide it into multiple image blocks. Calculate the average entropy of each image block based on the prediction confidence. Identify the image blocks with the largest and smallest average entropy as difficult and easy regions, respectively. S3: Divide the unlabeled image into multiple image pairs; for each image pair... ,use Simple region replacement In difficult regions, simple, strongly enhanced images are obtained, and the following methods are utilized: The part corresponding to the simple area in the pseudo tag, covering The pseudo-labels are obtained by identifying the positions of the original difficult regions in the pseudo-labels; using... Difficult area replacement From simple regions, difficult-to-enhance images are obtained, and the following methods are utilized: The part of the pseudo-tag corresponding to the difficult area is covered. The pseudo-labels correspond to the original simple regions in the pseudo-labels, and the pseudo-labels of the enhanced image are obtained. and ; S4: Input the simple strongly enhanced image into the student network to obtain the segmentation mask, and calculate the loss between the segmentation mask and the pseudo-label. The hard-enhanced image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the pseudo-label is calculated. ; S5: According to Calculate the loss of the current training step And based on the loss Update the parameters for the student network, and then update the parameters for the teacher network; Among them, the labeled image is the ultrasound image with the target region mask labeled, and the unlabeled image is the ultrasound image without the target region mask labeled; and Losses and The weight.

[0043] The following provides a further explanation of each step in each training step.

[0044] In step S1 of this embodiment, the loss between the segmentation mask and the segmentation label of the labeled image is... The standard cross-entropy (CE) is used for calculation, and the specific formula is as follows;

[0045] in, To label the batch size of the images, For student networks to annotate images Predicted segmentation mask, For labeled images The segmentation tag.

[0046] In this embodiment, steps S2-S3 target unlabeled images and achieve data augmentation by generating consistency regularization through bidirectional complementary generation of easy and difficult regions. Figure 1 As shown in (a); step S4 calculates the unlabeled loss based on the data augmented data. and ,like Figure 1 As shown in (b) of the example. Specifically, in S2, the prediction results of the teacher model are divided into blocks, and the average entropy of each image block is calculated. This average entropy reflects the ease with which the model learns the image block. Optionally, in step S2 of this embodiment, the first... Average entropy of image patches The calculation formula is as follows:

[0047] in, Indicates the total number of categories. and These represent the height and width of the image patch, respectively. Indicates the first The first image patch Each pixel belongs to the category The confidence level. For simple regions, the teacher network's predictions for that region will have a higher confidence level. It will be closer to 1, thus making the overall average entropy... Closer to 0, for difficult regions, the overall average entropy is... It is closer to 1, through the average entropy. It can determine the difficulty level of each block in the image. The greater the difficulty, the higher the average entropy value. This average entropy provides important guidance for subsequent complementary analysis of easy and difficult regions.

[0048] After obtaining the average entropy of each pixel block in the unlabeled image, step S2 of this embodiment selects the image blocks with the largest and smallest average entropy as difficult and easy blocks, respectively. To make reasonable use of the effective information surrounding the difficult and easy blocks, as a preferred embodiment, after determining the difficult and easy blocks, the system randomly extends outwards from the centers of the difficult and easy blocks to obtain difficult and easy regions, specifically including: The image patch with the largest average entropy is identified as the center of the hard region. A random-sized extension is then made outwards from this center on the unlabeled image, and the resulting extended region is taken as the final hard region. The hard region mask can then be obtained accordingly. ; The image patch with the minimum average entropy is identified as the center of a simple region. Randomly sized regions are then extended outwards from this center on the unlabeled image. These extended regions are then used as the final simple regions, and the corresponding simple region masks are obtained. ; In this case, after extension, the difficult region is the same size as the easy region; the size of the extended difficult region... satisfy , The size of the difficult area before extension. The size of the unlabeled image; To avoid introducing interference during the random extension process, and to set a preset ratio, in this embodiment, Similarly, the size of the extended simple region satisfy , The size of the difficult area before extension.

[0049] In this embodiment, after calculating the average entropy of the image patch based on the prediction confidence and identifying the regions with the maximum and minimum average entropy, a certain range is further randomly extended. This allows for the reasonable use of information around the region, thereby further improving the segmentation performance of the trained ultrasound image segmentation system.

[0050] After identifying the difficult and easy regions in each unlabeled image, step S3 of this embodiment will perform bidirectional swapping of difficult and easy regions within the same batch. Optionally, in this embodiment, the sum of the numbers will be... If two unlabeled images are grouped into one image pair, then the process of bidirectional swapping of easy and difficult regions can be represented by the following formula:

[0051]

[0052]

[0053]

[0054] Where BS represents the batch size. Indicates an index of unlabeled images; Indicates the first unlabeled image in the batch. Zhang Fei's unannotated image, and They represent the first Difficult and simple region masks for Zhang Fei's labeled images This indicates that the teacher network is for unlabeled images Predicted segmentation mask, " indicates coverage; This represents a simple, strongly enhanced image. express The segmentation pseudo-label; Representing images that are difficult to enhance. express The segmentation pseudo-labels.

[0055] This embodiment replaces the difficult regions in an image with simpler regions from another image, resulting in a relatively simple, strongly enhanced image and pseudo-label pair that includes more background and segmented target center regions. Meanwhile, by replacing simple regions in one image with difficult regions in another, a more challenging, heavily augmented image and pseudo-label pair were obtained, containing more segmented target boundary regions. .

[0056] Based on bidirectional complementarity of easy and difficult regions and In step S4 of this embodiment, the loss corresponding to the simple strongly enhanced image can be calculated. Loss corresponding to difficult image enhancement Optionally, this embodiment uses standard cross-entropy loss (CE) for calculation. and Based on the idea of ​​consistency regularization, the model's predictions for strongly enhanced images must be consistent with those for the original images in order to accurately perform unlabeled loss. and The calculation in this embodiment uses a threshold. To filter out unreliable pixels, only the teacher model network results with a confidence level of not less than [value missing]. Only pixels that meet the criteria will be considered as usable labels for calculating unlabeled loss. and , and The specific calculation expression is as follows:

[0057]

[0058] in, and These represent the teacher network and the student network, respectively, for Predicted segmentation mask, and These represent the teacher network and the student network, respectively, for Predicted segmentation mask, Represents cross-entropy loss, Indicates the size of the unlabeled batch.

[0059] In this embodiment, before calculating the segmentation loss of unlabeled images, pixels with low prediction confidence are removed, i.e., unreliable pixels are filtered out. This improves the accuracy of segmentation loss calculation and further enhances the segmentation effect of the trained ultrasound image segmentation system.

[0060] Step S5 in this embodiment will incorporate the annotation loss calculated in the preceding steps. and unlabeled loss and Calculate the total loss This embodiment argues that applying different weights to the loss values ​​obtained from streams of varying difficulty at different training stages is meaningful. In the early stages of training, images containing more background and the central region of the target can help the model quickly learn more stable segmentation results. In the later stages, the model's learning focus should shift to regions with poor segmentation, such as the target boundary, thereby further improving performance on weak tests. This aligns with the learning philosophy of courses: all learners begin with simple knowledge, mastering it before gradually tackling more difficult concepts. Therefore, this embodiment designs a progressive loss weighting. At the start of training, simple, strongly enhanced images are assigned larger weights, while difficult, strongly enhanced images are assigned smaller weights. As training progresses, the weights of simple, strongly enhanced images gradually decrease, while the weights of difficult, strongly enhanced images gradually increase. The weights of simple, strongly enhanced images are... And the weight of the image with difficulty They are as follows:

[0061]

[0062] in, Indicates the current training round number. This represents the total number of training epochs. As training progresses, the weights of simple images gradually decrease, while the weights of difficult images gradually increase. The training focus of the model gradually shifts from acquiring stable segmentation results to enhancing the segmentation performance of poorly segmented regions.

[0063] Based on the calculated loss This allows for parameter updates to both the student and teacher networks. In this embodiment, gradient propagation is performed only in the student network during training to update its parameters. And the parameters of the teacher network Then, based on the parameters of the student network The update is performed using an exponential moving average, and the relevant formula is as follows:

[0064] in, To update the teacher network parameters, To update the previous teacher network parameters; In this embodiment, the preset weighting coefficients are used. , This indicates the number of training steps.

[0065] In practical applications, the teacher network and student network can use any image segmentation model architecture. As a preferred implementation, in this embodiment, the student network specifically uses DINOV2-S as the encoder, and its structure is as follows: Figure 2 As shown. This embodiment uses DINOV2-S as the encoder, replacing the ResNet34 and ResNet50 encoders commonly used in ultrasound image segmentation. This embodiment can achieve better segmentation performance with a lower number of parameters.

[0066] Overall, this embodiment explicitly generates simplified and complex samples by bidirectionally replacing difficult and simple regions, and introduces a gradual weighting mechanism to dynamically adjust the contribution intensity of the two types of samples at each stage of training. At the same time, the cutting-edge DINOV2 encoder is introduced into the field of ultrasound image segmentation to replace the outdated ResNet encoder, which can effectively improve the segmentation performance of the trained ultrasound image segmentation system.

[0067] Example 2: A nerve block assist system, such as Figure 3 As shown, it includes: ultrasound equipment, data acquisition device, processing system and interactive device; Ultrasound equipment is used to acquire ultrasound images of patients during nerve blocks; One end of the data acquisition device is connected to the ultrasound equipment, and the other end is connected to the processing system. It is used to transmit the ultrasound images acquired by the ultrasound equipment to the processing system. The processing system is equipped with an ultrasound image segmentation system, which is trained by the ultrasound image segmentation system training method provided in Embodiment 1 above. One end of the processing system is connected to a data acquisition device, and the other end is connected to an interactive device. The system is used to input the ultrasound image to be segmented into the ultrasound image segmentation system to segment the ultrasound image and transmit the segmentation result to the interactive device. An interactive device, connected to the processing system, is used to display the received segmentation results.

[0068] Based on the ultrasound image segmentation system training method provided in Embodiment 1 above, the ultrasound image segmentation system deployed in the nerve block assist system provided in this embodiment can identify key structures in the image more quickly and accurately, providing effective guidance information for nerve block, thereby effectively assisting nerve block.

[0069] In this embodiment, the process of the processing system inputting the ultrasound image to be segmented into the ultrasound image segmentation system to segment the ultrasound image is as follows: Figure 4 As shown.

[0070] It is easy to understand that in practical applications, in order to enable the model to perform ultrasound image segmentation better, preprocessing may be included before inputting the patient's ultrasound image into the pre-trained ultrasound image segmentation system for nerve block; preprocessing includes removing irrelevant information, adjusting image size, etc.

[0071] To further reduce the difficulty of implementing nerve block, in this embodiment, the processing system is also used to perform visualization processing on the ultrasound image based on the segmentation mask obtained from the segmentation, and transmit the visualization result to the interactive device. Furthermore, the interactive device is also used to display visualization results.

[0072] In this embodiment, the visualization process includes: The regions where each anatomical structure is located are obtained based on the segmentation mask. Different colors are then used to cover the regions where different anatomical structures are located on the original patient ultrasound image to obtain the display image.

[0073] By overlaying the segmentation results of different anatomical structures, real-time visualization of the segmentation results can be achieved, facilitating the rapid and accurate localization of key structures during nerve block.

[0074] In this embodiment, the visualization process further includes: determining whether the segmentation result meets the preset conditions; if it does, determining that the original ultrasound image is a standard image; otherwise, determining that the original ultrasound image is a non-standard image; and calculating the degree of conformity between the original ultrasound image and the reference standard image. The reference standard image is an ultrasound image that provides optimal puncture guidance for nerve block; the preset conditions include: the presence of the target anatomical structure, and at least one of the following conditions: The outline of the target anatomical structure is clear; the position of the target anatomical structure is relatively stable; the size of the target anatomical structure is greater than the preset first threshold; the degree of conformity between the original ultrasound image and the reference standard image is greater than the preset second threshold.

[0075] In practical applications, the relevant thresholds can be set according to specific application requirements. Figure 5 The figures shown are two application examples of the ultrasound imaging visualization method for nerve block provided in this embodiment.

[0076] Based on the ultrasound image segmentation results, this embodiment significantly reduces the difficulty of implementing nerve block by attaching colors to key structures in the image and providing the degree of conformity between the current image and the standard image.

[0077] Furthermore, in the neural block assistive system provided in this embodiment, the interactive device is also used to receive the operator's instructions and transmit the received instructions to the processing system, thereby controlling the processing system; optionally, in this embodiment, the interactive device is a touch screen.

[0078] like Figure 6 As shown, the neurological blockade assistive system provided in this embodiment includes the following workflow: Step S01: Acquire ultrasound image data. The ultrasound image data originates from the ultrasound equipment and is transmitted to the processing system via a data acquisition device.

[0079] Step S02: Input the acquired ultrasound data into the processing system to obtain the recognition result.

[0080] Step S03: Display the recognition results on the touch screen. The touch screen receives the results transmitted from the processing system and displays them.

[0081] Step S04: The touch screen receives operator commands and inputs them into the processing system to control the processing system.

[0082] In practical applications, operators can interact with the software in the processing system through the touch display, and adjust the software's processing goals and output effects according to specific needs.

[0083] like Figure 7 As shown, step S01 specifically includes: Connect the data acquisition device to the ultrasound equipment; The data acquisition device acquires ultrasound image data in real time; the ultrasound equipment obtains real-time ultrasound images through an ultrasound probe and sends the ultrasound images to the data acquisition device through an external interface.

[0084] Step S02 specifically includes: The ultrasound image data acquired by the data acquisition device is transmitted to the processing system; the data is sent to the processing system via the acquisition device's transmission protocol for preprocessing and segmentation. The processing system performs preprocessing on ultrasound image data, such as removing irrelevant information and adjusting image size; The processed ultrasound image is input into the ultrasound image segmentation system for nerve block, which is built into the processing system. The model generates segmentation results through calculation.

[0085] The nerve block assist system provided in this embodiment can effectively process ultrasound image data, accurately segment the target structure, and realize the automatic identification and visualization of key anatomical structures during nerve block, providing intuitive surgical guidance during nerve block implementation.

[0086] The following analysis and verification of the beneficial effects that this invention can achieve are based on specific experimental results.

[0087] In the relevant experiments, the datasets used included the BUSI dataset, the TN3K dataset, and the UNS dataset. The BUSI dataset collected 780 breast ultrasound images from 600 women aged 25 to 75, including 133 normal images, 437 benign images, and 210 malignant images. We used only 647 images (benign and malignant) for the experiments. The TN3K dataset collected 3493 thyroid nodule ultrasound images from 2421 patients, divided into training and testing sets. The training set contained 2879 images, and the testing set contained 614 images. The UNS dataset collected a total of 11143 images, of which 2323 had normal annotations. This invention used only these 2323 images for the experiments.

[0088] Regarding image resolution, for the BUSI and TN3K datasets, this invention unifies the resolution to 500*500. For the UNS dataset, since its resolution is already unified, this invention maintains its original resolution of 580*420. In dataset partitioning, for the BUSI and UNS datasets, we divide the training, validation, and test sets in an 8:1:1 ratio. For the TN3K dataset, since a test set has already been created, this invention divides the training and validation sets in an 8:2 ratio.

[0089] Subsequently, for all three datasets, this invention follows the settings of commonly used semi-supervised image segmentation methods, using 1 / 8, 1 / 4, and 1 / 2 of the labeled images as labeled training sets, and the other images as unlabeled training sets for experiments. DICE SCORE is used as the evaluation metric during the experiments.

[0090] In the experiment, DPT was used as the segmentation model and DINOV2-S as the encoder. Based on existing methods, this invention set up two image enhancement schemes with different intensities during the experiment: weak enhancement and strong enhancement. Specifically, weak enhancement included random scaling of 0.5 to 2.0 times, random cropping of 336*336 pixels, and horizontal flipping with a probability of 0.5; strong enhancement included color dithering and the complementary use of easy and difficult regions provided by this invention (hereinafter referred to as strong-weak complementary CutMix).

[0091] For all datasets, the batch size for both labeled and unlabeled images is 4. The training optimizer is the AdamW optimizer with a weight decay of 0.001. For the DINOV2-Small encoder, the initial learning rate is 5e-6, and the learning rate is reduced using a poly scheduler. , Indicates the learning rate. Indicates the number of training steps. This represents the total number of training epochs. For all datasets, the number of training epochs is 100.

[0092] During validation, only simple interpolation was performed on the images to ensure they could be input into the model for inference. Similar to most existing methods, the relevant results were obtained from the EMA teacher network.

[0093] This invention compares relatively new methods, including three different encoders: ResNet34, ResNet50, and DINOV2-Small. Table 1 shows the encoders used in each method, the number of parameters trained on the BUSI dataset, and their performance. "Ours" indicates the segmentation method of the ultrasound image segmentation system trained based on this invention.

[0094]

[0095] As shown in Table 1, the method provided by this invention achieves the best performance under three different annotation ratios. Compared with the state-of-the-art (SOTA) method SFS based on ResNet, it improves performance by 0.84%, 2.06%, and 6.00% for 1 / 2, 1 / 4, and 1 / 8 annotation ratios, respectively. Compared with the latest UniMatchV2, which also uses the DINOV2-Small encoder, the method provided by this invention improves performance by 0.91%, 0.69%, and 2.03% for 1 / 2, 1 / 4, and 1 / 8 annotation ratios, respectively. Especially under the condition of fewer annotations, the method provided by this invention significantly surpasses the current state-of-the-art method. Moreover, it can be seen that even with fully supervised training on only a small number of annotated images, the performance obtained using the DINOV2-Small encoder basically exceeds most methods based on ResNet34 and ResNet50. This also proves that introducing the advanced encoder DINOV2 into the field of semi-supervised ultrasound image segmentation can further improve the segmentation performance of ultrasound image segmentation systems and reduce the number of parameters.

[0096] Some visualization results are as follows Figure 8 As shown, based on Figure 8 The results show that, under the same encoder and model structure, the method provided by this invention can always produce more accurate segmentation results and more accurate boundary prediction.

[0097] This invention further validates the role of progressive loss weights in the BUSI, TN3K, and UNS datasets with 1 / 4 annotation, as shown in Table 2. Here, "Sup" represents the fully supervised learning method, and "w / o DR" represents the unlabeled loss. and The weights are the same, and "wDR" represents the unlabeled loss. and The progressive loss proposed in this invention is adopted.

[0098]

[0099] As shown in Table 2, when the unlabeled loss... and Using the same weights, the performance was improved by 1.40%, 1.78%, and 1.09% compared to fully supervised learning methods in the three datasets, respectively. When using progressive weights, the performance was improved by 2.76%, 2.00%, and 1.51% compared to fully supervised learning methods, respectively. This shows that the complementary approach to easy and difficult regions provided by this invention can effectively improve the performance of semi-supervised learning, and its performance can even surpass that of fully supervised learning methods. Based on the complementary approach to easy and difficult regions, further using progressive weights can achieve a further performance improvement.

[0100] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for an ultrasound image segmentation system, characterized in that, include: Teacher and student networks for image segmentation are trained using labeled and unlabeled images. After training, the student network is used as an ultrasound image segmentation system. In each training step of each training round, the training includes the following steps: The labeled image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the segmentation label of the labeled image is calculated. ; Unlabeled images are input into the teacher network to obtain segmentation masks and prediction confidence. The segmentation masks are used as pseudo-labels for the unlabeled images. For each unlabeled image, it is divided into multiple image blocks. The average entropy of each image block is calculated based on the prediction confidence. The image blocks with the largest and smallest average entropy are identified as difficult and easy regions, respectively. Divide the unlabeled image into multiple image pairs; for each image pair... ,use Simple region replacement In difficult regions, simple, strongly enhanced images are obtained, and the following methods are utilized: The part corresponding to the simple area in the pseudo tag, covering The pseudo-labels are obtained by identifying the positions of the original difficult regions in the pseudo-labels; using... Difficult area replacement From simple regions, difficult-to-enhance images are obtained, and the following methods are utilized: The part of the pseudo-tag corresponding to the difficult area is covered. The pseudo-labels correspond to the original simple regions in the pseudo-labels, and the pseudo-labels of the enhanced image are obtained. and ; The simple strongly enhanced image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the pseudo-label is calculated. The hard-enhanced image is input into the student network to obtain a segmentation mask, and the loss between the segmentation mask and the pseudo-label is calculated. ; according to Calculate the loss of the current training step And based on the loss Update the parameters of the student network, and then update the parameters of the teacher network; Among them, the labeled image is the ultrasound image with the target region mask labeled, and the unlabeled image is the ultrasound image without the target region mask labeled; and Losses and The weight.

2. The training method for the ultrasound image segmentation system as described in claim 1, characterized in that, in, Indicates the current training round number. This indicates the total number of training rounds.

3. The training method for an ultrasound image segmentation system as described in claim 1 or 2, characterized in that, After identifying the image patches with the maximum and minimum average entropy as difficult and simple regions respectively, the process also includes: randomly extending the difficult and simple regions, and using the extended difficult and simple regions as the final difficult and simple regions. Randomly extend the difficult and easy regions, including: The image patch with the largest average entropy is identified as the center of the difficult region. A random-sized extension is then made outward from the center of the difficult region on the unlabeled image, and the extended region is taken as the final difficult region. The image patch with the smallest average entropy is identified as the center of a simple region. A random-sized extension is then made outward from the center of the simple region on the unlabeled image, and the extended region is taken as the final simple region. In this case, after extension, the difficult region is the same size as the easy region; the size of the extended difficult region... satisfy , The size of the difficult area before extension. The size of the unlabeled image. This is a preset ratio.

4. The training method for the ultrasound image segmentation system as described in claim 3, characterized in that, No. Average entropy of image patches The calculation formula is as follows: in, Indicates the total number of categories. and These represent the height and width of the image patch, respectively. Indicates the first The first image patch Each pixel belongs to the category The confidence level.

5. The training method for an ultrasound image segmentation system as described in claim 1 or 2, characterized in that, The student network uses DINOV2-S as the encoder.

6. A nerve block assist system, characterized in that, include: Ultrasonic equipment, data acquisition devices, processing systems, and interactive devices; The ultrasound device is used to acquire ultrasound images of the patient during nerve block. One end of the data acquisition device is connected to the ultrasound equipment, and the other end is connected to the processing system, for transmitting the ultrasound images acquired by the ultrasound equipment to the processing system; The processing system is equipped with an ultrasound image segmentation system, which is trained by the ultrasound image segmentation system training method according to any one of claims 1 to 5; one end of the processing system is connected to the data acquisition device, and the other end is connected to the interactive device, for inputting the ultrasound image to be segmented into the ultrasound image segmentation system to segment the ultrasound image, and transmitting the segmentation result to the interactive device. The interactive device, connected to the processing system, is used to display the received segmentation results.

7. The nerve block assist system as described in claim 6, characterized in that, The processing system is also used to perform visualization processing on the ultrasound image based on the segmentation mask obtained from the segmentation, and transmit the visualization result to the interactive device. Furthermore, the interactive device is also used to display the visualization results.

8. The nerve block assist system as described in claim 7, characterized in that, The visualization process includes: The regions where each anatomical structure is located are obtained based on the segmentation mask. Different colors are then used to cover the regions where different anatomical structures are located on the original patient ultrasound image to obtain the display image.

9. The nerve block assist system as described in claim 8, characterized in that, The visualization process also includes: determining whether the segmentation result meets the preset conditions; if it does, the original ultrasound image is determined to be a standard image; otherwise, the original ultrasound image is determined to be a non-standard image; and calculating the degree of conformity between the original ultrasound image and the reference standard image. The reference standard image is an ultrasound image that can provide the best puncture indication for nerve block.

10. The nerve block assist system as described in claim 9, characterized in that, The preset conditions include: the appearance of the target anatomical structure, and at least one of the following conditions: The outline of the target anatomical structure is clear; the position of the target anatomical structure is relatively stable; the size of the target anatomical structure is greater than the preset first threshold; the degree of conformity between the original ultrasound image and the reference standard image is greater than the preset second threshold.

Images