Multitask medical ultrasound image processing method, apparatus, and electronic device

Abstract

Embodiments of the present disclosure disclose a multi-task medical ultrasound image processing method, device and electronic equipment. A specific embodiment of the method comprises: batch division on a current medical ultrasound image set to be analyzed; feature extraction on each batch medical ultrasound image group in the batch medical ultrasound image group set to generate a batch feature group, thereby obtaining a batch feature group set; batch correction on each batch feature group in the batch feature group set to generate a batch corrected feature group; spatial optimization on each batch corrected feature in each batch corrected feature group in the batch corrected feature group set to generate an optimized batch corrected feature; and image segmentation on each optimized batch corrected feature group to generate a medical ultrasound image segmentation result group. The embodiment improves the adaptability of the model to new data sources while maintaining high segmentation accuracy, thereby providing reliable technical support for cross-institutional and cross-device medical ultrasound image analysis.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of image processing, and more specifically to multi-task medical ultrasound image processing methods, apparatuses, and electronic devices. Background Technology

[0002] Medical ultrasound image data from different batches, devices, or institutions often exhibit significant differences, which severely impact the performance and accuracy of downstream tasks such as ultrasound image segmentation or reconstruction. Currently, the approach to image processing of medical ultrasound images employs batch correction methods that are separate from deep learning models.

[0003] However, when using the above methods to process medical images, a common technical problem is that it is difficult to effectively utilize the inherent relationship between batch correction and image processing tasks, and it is difficult to achieve end-to-end training and optimization, which reduces the model's generalization ability and robustness on multi-source data, thus leading to a decrease in the accuracy and efficiency of image processing. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure propose multi-task medical ultrasound image processing methods, apparatuses, and electronic devices to address the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a multi-task medical ultrasound image processing method, the method comprising: dividing a current set of medical ultrasound images to be analyzed into batches to obtain batch medical ultrasound image groups; extracting features from each batch medical ultrasound image group in the batch medical ultrasound image group set to generate batch feature groups, thereby obtaining a batch feature group set; performing batch correction on each batch feature group in the batch feature group set to generate batch corrected feature groups, thereby obtaining a batch corrected feature group set; for each batch corrected feature group in the batch corrected feature group set, performing spatial optimization on each batch corrected feature in the batch corrected feature group set to generate optimized batch corrected features, thereby obtaining an optimized batch corrected feature group set; and performing image segmentation on each optimized batch corrected feature group to generate a medical ultrasound image segmentation result group set, thereby obtaining a medical ultrasound image segmentation result group set.

[0007] Secondly, some embodiments of this disclosure provide a multi-task medical ultrasound image processing apparatus, comprising: a partitioning unit configured to partition a current set of medical ultrasound images to be analyzed into batches to obtain batch medical ultrasound image sets; an extraction unit configured to extract features from each batch of medical ultrasound images in the batch medical ultrasound image sets to generate batch feature sets to obtain a batch feature set; a correction unit configured to perform batch correction on each batch feature set in the batch feature set to generate batch corrected feature sets to obtain a batch corrected feature set; an optimization unit configured to perform spatial optimization on each batch corrected feature set in the batch corrected feature set to generate optimized batch corrected features to obtain an optimized batch corrected feature set; and a segmentation unit configured to perform image segmentation on each optimized batch corrected feature set to generate a medical ultrasound image segmentation result set to obtain a medical ultrasound image segmentation result set.

[0008] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0009] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0010] The above embodiments of this disclosure have the following beneficial effects: the accuracy and efficiency of medical ultrasound image processing are improved through the multi-task medical ultrasound image processing method of some embodiments of this disclosure. Specifically, the reason why the results of related image processing models are not accurate enough is that existing batch correction methods are often separated from deep learning models, making it difficult to effectively utilize the inherent correlation between batch correction and image processing tasks, and making it difficult to achieve end-to-end training and optimization. Based on this, the multi-task medical ultrasound image processing method of some embodiments of this disclosure first divides the current medical ultrasound image set to be analyzed into batches to obtain batch medical ultrasound image sets. Second, feature extraction is performed on each batch medical ultrasound image set in the above batch medical ultrasound image set to generate batch feature sets, resulting in batch feature set sets. Here, a batch correction layer is introduced after feature extraction, which can effectively handle data differences in high-dimensional feature space. Then, batch correction is performed on each batch feature set in the above batch feature set to generate batch corrected feature sets, resulting in batch corrected feature set sets. Thus, seamless integration of batch correction and deep learning models can be achieved, enabling end-to-end training of the entire process and simplifying the deployment and usage of the model. Next, for each batch-corrected feature group in the aforementioned batch-corrected feature set, spatial optimization is performed on each batch-corrected feature within that group to generate optimized batch-corrected features, resulting in optimized batch-corrected feature sets. This minimizes the differences in feature space between medical images from different sources, thereby enhancing the model's generalization ability when faced with images from diverse sources. Finally, image segmentation is performed on each optimized batch-corrected feature group to generate a set of medical ultrasound image segmentation results, resulting in a set of medical ultrasound image segmentation results. By introducing a batch correction layer, not only is the performance of medical ultrasound image segmentation improved, but high segmentation accuracy is also maintained while enhancing the model's adaptability to new data sources, providing reliable technical support for cross-institutional and cross-device medical ultrasound image analysis. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of a multi-task medical ultrasound image processing method according to some embodiments of this disclosure; Figure 2 This is a batch division flowchart of the multi-task medical ultrasound image processing method according to the present disclosure; Figure 3This is a schematic diagram of the structure of a medical image processing model based on the multi-task medical ultrasound image processing method of this disclosure; Figure 4 This is a schematic diagram of the structure of some embodiments of the multi-task medical ultrasound image processing apparatus according to the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0013] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0014] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0015] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] Figure 1 This is a flowchart of a multi-task medical ultrasound image processing method according to some embodiments of the present disclosure, illustrating a flowchart 100 of some embodiments of the multi-task medical ultrasound image processing method according to the present disclosure. The multi-task medical ultrasound image processing method includes the following steps: Step 101: Divide the current set of medical ultrasound images to be analyzed into batches to obtain batch sets of medical ultrasound images.

[0020] In some embodiments, the execution subject (e.g., a computing device) of the multi-task medical ultrasound image processing method can divide the current set of medical ultrasound images to be analyzed into batches to obtain batch sets of medical ultrasound images.

[0021] Each medical ultrasound image in the aforementioned medical ultrasound image set can be from different batches, different devices, and different institutions. For example, the various medical ultrasound images in the set can be ultrasound images acquired by multiple institutions using different camera devices from different body parts and different stages of development in multiple patients. In practice, the various medical ultrasound images in the set can first be mixed, and then the mixed set can be batched to obtain batch medical ultrasound image groups. Each batch of medical ultrasound images in the batch set can contain medical ultrasound image data from multiple batches, multiple devices, and multiple institutions. The flowchart corresponding to the batching process is as follows. Figure 2 As shown. In Figure 2 In this system, each square corresponds to a medical ultrasound image data point, and squares of different colors can represent medical ultrasound image data from different batches, devices, or institutions.

[0022] Step 102: Extract features from each batch of medical ultrasound images in the above batch medical ultrasound image set to generate batch feature sets, thus obtaining a batch feature set.

[0023] In some embodiments, the execution entity can perform feature extraction on each batch of medical ultrasound images in the batch medical ultrasound image set to generate batch feature sets, thus obtaining a batch feature set. In practice, firstly, for each batch of medical ultrasound images in the batch medical ultrasound image set, features can be extracted from each batch of medical ultrasound images in the batch medical ultrasound image set based on a preset encoder to generate batch features, thus obtaining a batch feature set. The batch features in the batch feature set can be obtained by extracting features from the batch medical ultrasound images in the batch medical ultrasound image set. The encoder can be a multi-layer convolutional neural network. For example, the encoder can include 5 encoding layers, each containing two 3×3 convolutional layers and two 2×2 max-pooling layers. Then, the obtained batch feature sets can be determined as the batch feature set.

[0024] As an example, the five coding layers of the encoder described above can downsample batches of medical ultrasound images to obtain coded feature maps of sizes 512×512, 256×256, 128×128, 64×64, and 32×32, respectively. The final 32×32 coded feature map is the batch feature.

[0025] Step 103: Perform batch correction on each batch feature group in the above batch feature group set to generate a batch correction feature group, thus obtaining a batch correction feature group set.

[0026] In some embodiments, the execution entity may perform batch correction on each batch feature group in the batch feature group set to generate a batch correction feature group, thereby obtaining a batch correction feature group set.

[0027] In practice, a pre-defined batch correction algorithm can be used to perform batch correction on each batch feature group in the aforementioned batch feature group set to generate batch-corrected feature groups, thus obtaining a batch-corrected feature group set. The aforementioned batch correction algorithm can be either the BBKNN (Batch Balanced K-Nearest Neighbors) algorithm or the Combat batch correction algorithm.

[0028] Each batch feature group in the aforementioned batch feature set can correspond to a batch sequence number. Each batch feature in a batch feature group can correspond to a sample index number. Each batch feature in the aforementioned batch feature set can be represented by feature maps with multiple different channels. Each element in the aforementioned feature map can correspond to a spatial location index number. Each batch feature in the aforementioned batch feature set can include multiple sub-channel features. The number of sub-channel features included in each batch feature set is the same. Each of the aforementioned multiple sub-channel features can correspond to a feature channel sequence number. The various sub-channel features in the aforementioned multiple sub-channel features can be feature maps of the same size but with different specific element values.

[0029] In another implementation, the aforementioned execution entity may perform the following processing steps for each batch feature group in the aforementioned batch feature group set: The first step is to perform feature standardization on each batch feature in the aforementioned batch feature group to generate standard batch features, thus obtaining a standard batch feature group. In practice, for each batch feature in the aforementioned batch feature group, a preset standardization algorithm can be used to standardize the batch features to obtain standard batch features. Then, the combination of the obtained standard batch features can be determined as the standard batch feature group. The aforementioned standardization algorithm can be either the Z-Score standardization algorithm or the Min-Max standardization algorithm.

[0030] The first step mentioned above may include: The first sub-step involves determining the batch mean for each batch feature in the aforementioned batch feature group. In practice, since each batch feature includes multiple sub-channel features, the channel batch mean for each sub-channel feature with the same feature channel number can be determined using the following formula.

[0031] .

[0032] in, This represents the average value of each batch in the channel. This represents the feature value at the j-th position of the k-th channel of the i-th sample in the b-th batch. i is the sample index, j is the spatial location, k is the feature channel, and b is the batch. This represents the product of the number of samples and the number of spatial locations in the b-th batch.

[0033] Then, the average value of each channel batch can be determined as the batch average. The batch average can be expressed as: Where n represents the number of sub-channel features, i.e., the number of feature channels.

[0034] The second sub-step involves determining the batch variance corresponding to each batch feature in the aforementioned batch feature group based on the batch mean. In practice, the channel batch variance of each sub-channel feature with the same feature channel number can be determined using the following formula.

[0035] .

[0036] in, This represents the variance of the channel batch.

[0037] The batch variance can then be determined by combining the determined batch variances of each channel. This batch variance can be expressed as: .

[0038] The third sub-step involves standardizing each batch feature in the batch feature group based on the batch mean and batch variance mentioned above, to generate standard batch features and thus a standard batch feature group. In practice, for each batch feature in the batch feature group, the following formula can be used to standardize each element of the batch feature to obtain the standard batch feature.

[0039] .

[0040] in, This represents the standardized feature of the j-th position in the k-th channel of the i-th sample in the b-th batch. This represents a preset constant (e.g., 1e-5) to prevent division by zero.

[0041] The second step involves performing an affine transformation on each standard batch feature in the aforementioned standard batch feature group to generate affine transformation features, resulting in an affine transformation feature group, which serves as the batch correction feature group. In practice, for each standard batch feature in the aforementioned standard batch feature group, a linear mapping can be performed on the aforementioned standard batch feature to generate an affine transformation feature.

[0042] The second step mentioned above may include the following sub-steps: The first sub-step involves determining the one-hot encoded vector corresponding to each batch of medical ultrasound images in the aforementioned batch medical ultrasound image set, thus obtaining a set of one-hot encoded vectors. Each batch of medical ultrasound images in the aforementioned batch medical ultrasound image set corresponds to one one-hot encoded vector. Therefore, the aforementioned standard batch feature groups also have corresponding one-hot encoded vectors.

[0043] As an example, suppose there are 3 batches of medical ultrasound images in the batch medical ultrasound image set. Then the one-hot encoded vectors corresponding to each batch of medical ultrasound images are [0,0,1], [0,1,0], and [1,0,0].

[0044] The second sub-step involves performing the following steps for each standard batch feature in the aforementioned standard batch feature group: 1. The one-hot encoded vectors corresponding to the standard batch features in the above one-hot encoded vector set are determined as the target batch vectors.

[0045] 2. Perform a first linear mapping on the target batch vector to obtain a first mapped vector. In practice, a preset first linear layer can be used to perform a first linear mapping on the target batch vector to obtain the first mapped vector, expressed as: .

[0046] in, This represents the first mapping vector. This indicates the first linear layer. This represents the target batch vector.

[0047] 3. Perform a second linear mapping on the target batch vector to obtain a second mapped vector. This can be achieved using a pre-defined second linear layer. Both the first and second linear layers can be fully connected layers. The first and second linear layers can have the same structure but different parameter values.

[0048] 4. Based on the first mapping vector and the second mapping vector described above, perform an affine transformation on the standard batch features to obtain the affine transformation features. In practice, for the feature value of each element in the standard batch features, the affine transformation can be performed on the elements using the following formula to obtain the affine transformation features.

[0049] .

[0050] in, This represents the transformed feature value of the j-th position in the k-th channel of the i-th sample in the b-th batch. This represents the second mapping vector.

[0051] In another alternative implementation, the aforementioned execution entity can perform an affine transformation on each standard batch feature in the aforementioned standard batch feature group through the following steps to generate an affine transformation feature: The first step is to determine the state vector corresponding to each batch feature group in the aforementioned batch feature group set, thus obtaining a state vector set. In practice, for each batch feature group in the aforementioned batch feature group set, the batch statistical vector corresponding to the aforementioned batch feature group can first be determined. The element values ​​of the aforementioned batch statistical vector can be the channel batch variance for each feature channel. The dimension of the aforementioned batch statistical vector can be n-dimensional, where n represents the number of feature channels.

[0052] Secondly, the distribution difference between the aforementioned batch feature groups and a preset reference batch feature group can be determined using KL divergence. Specifically, firstly, the probability distribution of the aforementioned batch feature groups and the probability distribution of the preset reference batch feature groups can be determined using a preset activation function. The activation function can be the Softmax function. Secondly, the distribution difference between the probability distribution of the aforementioned batch feature groups and the probability distribution of the preset reference batch feature groups can be determined using the following formula.

[0053] .

[0054] in, This represents the distribution difference value. P represents the probability distribution of the above batch characteristic group. Q represents the probability distribution of the above reference batch characteristic group. This represents the probability distribution of the i-th batch feature in the batch feature group. This represents the probability distribution of the i-th reference batch feature in the reference batch feature group. N represents the number of batch features in the batch feature group.

[0055] Next, the batch statistical vector and the distribution difference value can be concatenated to obtain the state vector. The state vector can be n+1 dimensional.

[0056] The second step is to determine the state vector corresponding to the standard batch feature group based on the aforementioned state vector set. In practice, the state vector of the batch feature group corresponding to the standard batch feature group can be used as the state vector corresponding to the standard batch feature group.

[0057] The third step involves performing the following reinforcement learning steps based on the pre-defined initial policy network: 1. Based on the initial policy network described above, determine the first mapping vector and the second mapping vector corresponding to the state vector. The initial policy network may include a fully connected layer. The initial policy network can output a mapping vector matrix based on the state vector. The first column of the mapping vector matrix can be the first mapping vector, and the second column can be the second mapping vector.

[0058] 2. Based on the first mapping vector and the second mapping vector, perform an affine transformation on each standard batch feature in the standard batch feature group to generate initial affine transformation features, thus obtaining the initial affine transformation feature group. In practice, the operation of obtaining the affine transformation features in step 4 "Affine Transformation" of the optional scheme above can be used to perform an affine transformation on each standard batch feature in the standard batch feature group based on the first mapping vector and the second mapping vector to obtain the initial affine transformation feature group. Further details are omitted here.

[0059] 3. Based on the preset reference batch feature group, generate the reward value corresponding to the initial affine transformation feature group. In practice, firstly, the distribution difference value between the initial affine transformation feature group and the reference batch feature group can be determined. Then, through steps 104 and 105 below, the image segmentation result corresponding to each affine transformation feature in the initial affine transformation feature group can be determined to obtain the image segmentation result group. Here, each image segmentation result in the image segmentation result group corresponds to an image segmentation ground value. Next, through a preset loss function, the loss value between each image segmentation result in the image segmentation result group and the corresponding image segmentation ground value, as well as the mean loss value among the various loss values, can be determined. The loss function can be the cross-entropy loss function. Secondly, the negative of the sum of the distribution difference value and the mean loss value can be determined as the reward value. 4. Based on the aforementioned reward values, update the initial policy network to obtain the updated policy network. In practice, a preset policy optimization algorithm can be used to train the initial policy network using reinforcement learning based on the aforementioned reward values ​​to obtain the updated policy network. The aforementioned policy optimization algorithm can be the Proximal Policy Optimization (PPO) algorithm.

[0060] Fourth, in response to determining that the initial affine transformation feature set satisfies the preset transformation conditions, the initial affine transformation feature set is determined as an affine transformation feature set. The aforementioned transformation conditions can be that the distribution difference value and the mean loss value corresponding to the initial affine transformation feature set are both less than preset distribution difference thresholds and loss thresholds. In practice, when the distribution difference value and the mean loss value corresponding to the initial affine transformation feature set are both less than the preset distribution difference thresholds and loss thresholds, the initial affine transformation feature set can be determined as an affine transformation feature set. The aforementioned distribution difference value can be a preset value, and is not specifically limited here. The aforementioned loss threshold can be a preset value, and is not specifically limited here.

[0061] Fifth, in response to the determination that the initial transformation feature set does not meet the preset transformation conditions, the updated policy network is determined as the initial policy network, and the reinforcement learning steps described above are executed again. In practice, when the distribution difference value corresponding to the initial affine transformation feature set is greater than the distribution difference threshold or the mean loss is greater than the loss threshold, the updated policy network can be determined as the initial policy network, and the reinforcement learning steps described above can be executed again.

[0062] First, the distribution difference between each batch feature group and a preset reference batch feature group is represented using KL divergence. Then, the distribution difference is combined with the batch variance of the batch feature group to construct a state vector reflecting the statistical characteristics of the batch feature group. This allows for dynamic determination of the state corresponding to the batch feature group based on the data and distribution contained in each batch feature, avoiding errors introduced by fixed one-hot encoding representation. Next, the state vector is input into a policy network to output a first mapping vector and a second mapping vector. The policy network is dynamically trained using a reward function, enabling it to autonomously learn optimization strategies for the first and second mapping vectors. When the number of medical ultrasound image samples is small or when there are non-linear distribution differences between batches, the policy network can adaptively learn complex batch correction rules through non-linear mapping, outputting more accurate mapping vectors, thereby obtaining more accurate transformation features and improving the quality of batch correction.

[0063] Step 104: For each batch correction feature group in the above batch correction feature group set, perform spatial optimization on each batch correction feature in the above batch correction feature group to generate optimized batch correction features, and obtain optimized batch correction feature groups.

[0064] In some embodiments, the execution entity may perform batch correction on each batch feature group in the batch feature group set to generate a batch-corrected feature group, thus obtaining a batch-corrected feature group set. In practice, for each batch-corrected feature group in the batch-corrected feature group set, a preset feature alignment algorithm can be used to optimize the feature space of each batch-corrected feature in the batch-corrected feature group to generate optimized features, thus obtaining an optimized feature group. The feature alignment algorithm may be a Mutual Nearest Neighbors (MNN) algorithm.

[0065] In practice, the aforementioned implementing entity can perform spatial optimization on each batch correction feature in the aforementioned batch correction feature group through the following steps to generate optimized batch correction features: The first step is to perform global dimensionality reduction on each batch-corrected feature group in the aforementioned batch-corrected feature set to generate low-dimensional features, thus obtaining a low-dimensional feature set. In practice, for each batch-corrected feature group in the aforementioned batch-corrected feature set, a preset feature dimensionality reduction algorithm can be used to perform global dimensionality reduction on each batch-corrected feature in the aforementioned batch-corrected feature set to generate low-dimensional features, thus obtaining a low-dimensional feature set. The feature dimensionality reduction algorithm can be global pooling or the Uniform Manifold Approximation and Projection (UMAP) algorithm. Here, each low-dimensional feature in each of the obtained low-dimensional feature sets can be a feature with 1 channel.

[0066] The second step involves generating a feature graph structure based on the obtained low-dimensional feature groups. This feature graph structure includes a node set and an edge set. The feature graph structure can be a graph-like data structure. Each node in the node set represents a low-dimensional feature within the low-dimensional feature group. Each edge in the edge set connects two nodes in the node set. Each edge in the edge set can have a corresponding weight. In practice, for each low-dimensional feature group, each low-dimensional feature can be used as a node in the feature graph structure to generate a node set. Next, for each node in the node set, the m nearest neighbors can be identified as its neighbors. Then, each node in the node set can be connected to its corresponding neighbors to generate an initial edge set. The weight of each edge in the initial edge set can be the distance between the two corresponding nodes. Finally, attention features can be extracted from the initial graph structure consisting of the node set and the initial edge set using a pre-defined Graph Attentional Layer (GAT) to generate attention weights corresponding to each initial edge in the initial edge set, thus obtaining the edge set. The weights corresponding to each edge in the edge set can be attention weights.

[0067] In practice, the larger the attention weight of an edge, the stronger the association between the two nodes connected by that edge, meaning that the feature semantics between the two medical ultrasound images are more similar.

[0068] The third step is to determine the batch feature stability corresponding to each low-dimensional feature group in the above-mentioned feature map structure, thereby obtaining the batch feature stability set. In practice, for each low-dimensional feature group, the batch data size and batch cluster compactness can be determined first. The batch data size can be the number of low-dimensional features in the low-dimensional feature group. The batch cluster compactness can be the average sum of the edge weights between the low-dimensional features in the low-dimensional feature group. Then, a weighted summation can be performed on the batch data size and the batch cluster compactness to generate the batch feature stability corresponding to the low-dimensional feature group. Here, the weights used in the weighted summation process can be hyperparameters, and no specific limitations are imposed here.

[0069] Specifically, for each low-dimensional feature in the aforementioned low-dimensional feature group, based on the aforementioned feature graph structure, the edges connecting the nodes corresponding to the low-dimensional feature can be determined, and the weights of these edges can be summed to obtain the sum of edge weights corresponding to the low-dimensional feature. Then, the average value among the obtained sums of edge weights can be determined as the batch cluster compactness. The fourth step involves identifying the low-dimensional feature groups corresponding to the batch feature stability sets that satisfy the preset stability conditions as low-dimensional control feature groups, and identifying the batch-corrected feature groups corresponding to the low-dimensional feature groups as high-dimensional control feature groups. The aforementioned stability condition can be the batch feature stability with the highest corresponding value in the aforementioned batch feature stability set.

[0070] Fifth, based on the batch feature stability set and the various low-dimensional feature groups mentioned above, generate a low-dimensional feature group sequence. Each low-dimensional feature group in the sequence is arranged in descending order of its corresponding batch feature stability. In practice, the low-dimensional feature groups can be sorted according to the descending order of batch feature stability in the batch feature stability set to obtain the low-dimensional feature group sequence.

[0071] Step 6: For each low-dimensional feature group in the above low-dimensional feature group sequence other than the above low-dimensional control feature group, perform the following optimization steps: 1. Based on the above feature map structure, determine the nearest neighbor feature pair set between the above low-dimensional feature group and the above low-dimensional contrast feature group. Each nearest neighbor feature pair in the above nearest neighbor feature pair set includes a node pair and edge weights. Each node pair can include two nodes. For each nearest neighbor feature pair in the above nearest neighbor feature pair set, one node can be a low-dimensional feature from the above low-dimensional feature group, and the other node can be a low-dimensional contrast feature from the above low-dimensional contrast feature group. In practice, firstly, for each low-dimensional contrast feature in the low-dimensional contrast feature group, the top *a* low-dimensional features with the largest edge weights between them and the low-dimensional contrast feature are determined as the nearest neighbor feature set corresponding to the low-dimensional contrast feature. Next, for each low-dimensional feature in the above low-dimensional feature group, the top *a* low-dimensional contrast features with the largest edge weights between them and the low-dimensional contrast feature are determined as the nearest neighbor contrast feature set corresponding to the low-dimensional feature. Next, for each low-dimensional contrast feature in the aforementioned low-dimensional contrast feature group, the following steps are performed: For each low-dimensional feature in the aforementioned low-dimensional feature group, if the aforementioned low-dimensional contrast feature exists in the nearest neighbor contrast feature set corresponding to the aforementioned low-dimensional feature, and the aforementioned low-dimensional feature exists in the nearest neighbor feature set corresponding to the aforementioned low-dimensional contrast feature, then the aforementioned low-dimensional contrast feature and the aforementioned low-dimensional feature are determined as a nearest neighbor feature pair. Finally, based on each determined nearest neighbor feature pair, a nearest neighbor feature pair set is generated.

[0072] 2. Based on the aforementioned nearest neighbor feature pair set, the aforementioned high-dimensional control feature set, and the batch-corrected feature set corresponding to the aforementioned low-dimensional feature set, generate a weighted control feature. In practice, this can be achieved using the following formula: .

[0073] in, This indicates the weighted comparison feature. This represents the edge weight of the j-th nearest neighbor feature pair in the nearest neighbor feature pair set. This represents the batch-corrected feature corresponding to the low-dimensional feature in the j-th nearest neighbor feature pair. This represents the high-dimensional control feature corresponding to the low-dimensional control feature in the j-th nearest neighbor feature pair.

[0074] 3. The sum of each batch-corrected feature in the batch-corrected feature group corresponding to the above low-dimensional feature group and the above weighted control feature is determined as the optimized batch-corrected feature, thus obtaining the optimized batch-corrected feature group.

[0075] First, dimensionality reduction of high-dimensional features allows for nearest-neighbor feature matching using low-dimensional features, thus improving the accuracy of neighborhood search. Next, a feature map structure is constructed, enabling graph neural networks to capture the strength of associations between nodes. Second, based on the feature map structure, batch feature stability is generated for each batch, allowing the batch with the highest stability to be designated as the control batch. Next, feature alignment is performed on each batch sequentially based on the control batch. Specifically, the set of nearest-neighbor feature pairs is first determined using the edge weights in the feature map structure. This reduces biases arising from distance-based nearest-neighbor judgments, avoiding the selection of close but biologically unrelated nearest-neighbor feature pairs. Then, when generating weighted control features, the weight differences of each nearest-neighbor feature pair are considered, ensuring that features with higher relevance contribute more to the results, improving the accuracy of the generated weighted control features. Finally, the high-dimensional features are aligned in the feature space using the weighted control features. This preserves key biological features, thereby improving the accuracy of medical ultrasound image segmentation.

[0076] Step 105: Perform image segmentation on each optimized batch correction feature group to generate a medical ultrasound image segmentation result set, thus obtaining a medical ultrasound image segmentation result set.

[0077] In some embodiments, the aforementioned execution entity can perform image segmentation on each optimized batch correction feature group to generate a medical ultrasound image segmentation result set. In practice, image segmentation can be performed on each optimized batch correction feature group based on a preset image segmentation decoder to generate a medical ultrasound image segmentation result set. The aforementioned image segmentation decoder can be a DeepLabV3 model.

[0078] In practice, the aforementioned implementing entity can perform image segmentation on each optimized batch of corrected feature groups through the following steps to generate a medical ultrasound image segmentation result group: The first step involves multi-scale feature decoding of each optimized batch-corrected feature in the aforementioned optimized batch-corrected feature group to generate decoded features, thus obtaining a decoded feature group. The decoded features in this decoded feature group correspond one-to-one with the optimized batch-corrected features in the aforementioned optimized batch-corrected feature group. In practice, a pre-defined decoder can be used to perform multi-scale feature decoding on the optimized batch-corrected features to obtain the decoded features. The decoder can contain 5 decoding layers, each consisting of one 2×2 transposed convolutional layer and two 3×3 convolutional layers.

[0079] As an example, the first layer of the decoder can perform transpose convolution and convolution operations on the optimized batch correction features of size 32×32 to obtain a decoded feature map of size 64×64. Then, the 64×64 encoded feature map obtained from the encoder can be concatenated with the decoded feature map to obtain a fused feature map. This helps preserve low-level features and improves the details of reconstruction and segmentation. The second layer of the decoder can perform transpose convolution and convolution operations on the fused feature map to obtain a decoded feature map of size 128×128. Finally, the last layer of the decoder can output a feature map of the same size as the original medical ultrasound image.

[0080] The second step is to determine the medical ultrasound image segmentation result group corresponding to the aforementioned decoding feature group. Each medical ultrasound image segmentation result in the aforementioned medical ultrasound image segmentation result group can be a semantic segmentation image obtained after image segmentation of the medical ultrasound image segmentation result. In practice, a preset fully connected layer can be used to determine the medical ultrasound image segmentation result corresponding to each decoding feature in the aforementioned decoding feature group, thereby generating the medical ultrasound image segmentation result group.

[0081] Optionally, image reconstruction is performed on each optimized batch correction feature group in the obtained optimized batch correction feature groups to generate a medical ultrasound image reconstruction result set, thus obtaining a medical ultrasound image reconstruction result set.

[0082] In some embodiments, the execution entity can perform image reconstruction on each optimized batch correction feature group in the obtained optimized batch correction feature groups to generate a medical ultrasound image reconstruction result set. The medical ultrasound image reconstruction result contained in each medical ultrasound image reconstruction result set can be a reconstructed image obtained after reconstructing the medical ultrasound image data. In practice, a preset image reconstruction decoder can be used to perform image reconstruction on each optimized batch correction feature group in the obtained optimized batch correction feature groups to generate a medical ultrasound image reconstruction result set. The image reconstruction decoder can be a U-Net model.

[0083] In practice, by using a dual-decoder structure for data reconstruction and image segmentation tasks respectively, it is possible to simultaneously learn the common features and task-specific features of medical ultrasound image data, thereby improving the model's generalization ability and task performance.

[0084] Optionally, the aforementioned execution entity can integrate the batch partitioning, feature extraction, batch correction, feature space optimization, image segmentation, and image reconstruction steps 101 to 105 as a medical ultrasound image processing model. A schematic diagram of the medical ultrasound image processing model is shown below. Figure 3 As shown, the above-mentioned medical ultrasound image processing model can be trained end-to-end. In this end-to-end training, a joint loss function can be used to train the medical ultrasound image processing model. The joint loss function can be a weighted sum of the image segmentation loss and the image reconstruction loss. The image reconstruction loss (L_recon) can use mean squared error (MSE) or structural similarity index (SSIM). The image segmentation loss (L_seg) can use Dice coefficient loss or cross-entropy loss. The joint loss can be L_total = a × L_recon + b × L_seg, where a and b are weight hyperparameters.

[0085] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a multi-task medical ultrasound image processing device, which are similar to... Figure 1 Corresponding to the method embodiments shown, this multi-task medical ultrasound image processing device can be specifically applied to various electronic devices.

[0086] like Figure 4As shown, a multi-task medical ultrasound image processing apparatus 400 in some embodiments includes: a division unit 401, an extraction unit 402, a correction unit 403, an optimization unit 404, and a segmentation unit 405. The system comprises the following components: a partitioning unit 401, configured to partition the current set of medical ultrasound images to be analyzed into batches, resulting in batch sets of medical ultrasound images; an extraction unit 402, configured to extract features from each batch of medical ultrasound images in the batch set to generate batch feature groups, resulting in a batch feature group set; a correction unit 403, configured to perform batch correction on each batch feature group in the batch feature group set to generate batch corrected feature groups, resulting in a batch corrected feature group set; an optimization unit 404, configured to perform spatial optimization on each batch corrected feature group in the batch corrected feature group set to generate optimized batch corrected features, resulting in an optimized batch corrected feature group; and a segmentation unit 405, configured to perform image segmentation on each optimized batch corrected feature group to generate a medical ultrasound image segmentation result set, resulting in a medical ultrasound image segmentation result group set.

[0087] It is understandable that the units described in the multi-task medical ultrasound image processing device 400 are similar to those in the reference device. Figure 1 The steps described in the method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the multi-task medical ultrasound image processing device 400 and the units contained therein, and will not be repeated here.

[0088] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0089] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage device 508 into a random access memory 503. The random access memory 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0090] Typically, the following devices can be connected to the input / output interface 505: input devices 506 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0091] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a read-only memory 502. When the computer program is executed by the processing device 501, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0092] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0093] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0094] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: batch-divide the currently analyzed set of medical ultrasound images to obtain batch medical ultrasound image groups; extract features from each batch of medical ultrasound images in the batch medical ultrasound image groups to generate batch feature groups, obtaining a batch feature group set; perform batch correction on each batch feature group in the batch feature group set to generate batch corrected feature groups, obtaining a batch corrected feature group set; for each batch corrected feature group in the batch corrected feature group set, perform spatial optimization on each batch corrected feature in the batch corrected feature group to generate optimized batch corrected features, obtaining an optimized batch corrected feature group; and perform image segmentation on each optimized batch corrected feature group to generate a medical ultrasound image segmentation result set, obtaining a medical ultrasound image segmentation result group set.

[0095] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0098] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A multi-task medical ultrasound image processing method, characterized in that, include: The current set of medical ultrasound images to be analyzed is divided into batches to obtain batch sets of medical ultrasound images; Feature extraction is performed on each batch of medical ultrasound images in the batch medical ultrasound image set to generate batch feature sets, thus obtaining a batch feature set; Batch correction is performed on each batch feature group in the batch feature group set to generate batch corrected feature groups, thus obtaining a batch corrected feature group set; For each batch correction feature group in the batch correction feature group set, spatial optimization is performed on each batch correction feature in the batch correction feature group to generate optimized batch correction features, thus obtaining optimized batch correction feature groups. Image segmentation is performed on each optimized batch of corrected feature groups to generate medical ultrasound image segmentation result sets, thus obtaining a medical ultrasound image segmentation result set.

2. The multi-task medical ultrasound image processing method according to claim 1, characterized in that, The step of performing batch correction on each batch feature group in the batch feature group set to generate a batch-corrected feature group includes: For each batch feature group in the batch feature group set, perform the following processing steps: Each batch feature in the batch feature group is subjected to feature standardization processing to generate standard batch features, thus obtaining a standard batch feature group; Affine transformation is performed on each standard batch feature in the standard batch feature group to generate affine transformation features, resulting in an affine transformation feature group, which serves as the batch correction feature group.

3. The multi-task medical ultrasound image processing method according to claim 2, characterized in that, The step of performing feature standardization processing on each batch feature in the batch feature group to generate standard batch features includes: Determine the batch mean value corresponding to each batch feature in the batch feature group; Based on the batch mean, determine the batch variance corresponding to each batch feature in the batch feature group; Based on the batch mean and the batch variance, feature standardization is performed on each batch feature in the batch feature group to generate standard batch features, thus obtaining the standard batch feature group.

4. The multi-task medical ultrasound image processing method according to claim 2, characterized in that, The step of performing an affine transformation on each standard batch feature in the standard batch feature group to generate an affine transformation feature includes: Determine the one-hot encoded vector corresponding to each batch of medical ultrasound images in the batch medical ultrasound image set to obtain the one-hot encoded vector set. For each standard batch feature in the standard batch feature group, perform the following steps: The one-hot encoded vectors in the set of one-hot encoded vectors that correspond to the standard batch features are determined as the target batch vectors; Perform a first linear mapping on the target batch vector to obtain a first mapping vector; A second linear mapping is performed on the target batch vector to obtain a second mapped vector; Based on the first mapping vector and the second mapping vector, an affine transformation is performed on the standard batch features to obtain affine transformation features.

5. The multi-task medical ultrasound image processing method according to claim 4, characterized in that, The step of performing image segmentation on each optimized batch of corrected feature groups to generate a medical ultrasound image segmentation result group includes: Multi-scale feature decoding is performed on each optimized batch correction feature in the optimized batch correction feature group to generate decoded features, thus obtaining the decoded feature group; Determine the medical ultrasound image segmentation result group corresponding to the decoded feature group.

6. The multi-task medical ultrasound image processing method according to claim 1, characterized in that, The method further includes: Image reconstruction is performed on each optimized batch correction feature group in the obtained optimized batch correction feature group to generate a medical ultrasound image reconstruction result set.

7. The multi-task medical ultrasound image processing method according to claim 2, characterized in that, The step of performing an affine transformation on each standard batch feature in the standard batch feature group to generate an affine transformation feature includes: Determine the state vector corresponding to each batch feature group in the batch feature group set to obtain a state vector set; Based on the state vector set, determine the state vector corresponding to the standard batch feature group; Based on the pre-defined initial policy network, the following reinforcement learning steps are performed: Based on the initial policy network, determine the first mapping vector and the second mapping vector corresponding to the state vector; Based on the first mapping vector and the second mapping vector, an affine transformation is performed on each standard batch feature in the standard batch feature group to generate an initial affine transformation feature, thus obtaining an initial affine transformation feature group. Based on the preset reference batch feature group, the reward value corresponding to the initial affine transformation feature group is generated; The initial policy network is updated based on the reward value to obtain the updated policy network; In response to determining that the initial affine transformation feature set satisfies a preset transformation condition, the initial affine transformation feature set is determined as an affine transformation feature set; In response to the determination that the initial transformation feature set does not meet the preset transformation conditions, the updated policy network is determined as the initial policy network, and the reinforcement learning step is executed again.

8. A multi-task medical ultrasound image processing device, characterized in that, include: The partitioning unit is configured to partition the current set of medical ultrasound images to be analyzed into batches, resulting in batch sets of medical ultrasound images. The extraction unit is configured to extract features from each batch of medical ultrasound images in the batch medical ultrasound image set to generate batch feature sets, thus obtaining a batch feature set. The correction unit is configured to perform batch correction on each batch feature group in the batch feature group set to generate a batch corrected feature group, thereby obtaining a batch corrected feature group set. The optimization unit is configured to perform spatial optimization on each batch correction feature in each batch correction feature group in the batch correction feature group set to generate optimized batch correction features, thereby obtaining optimized batch correction feature groups. The segmentation unit is configured to perform image segmentation on each optimized batch of corrected feature groups to generate a medical ultrasound image segmentation result set.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images