Ultrasonic image segmentation method, system, medium, and terminal based on portable devices

Abstract

This application discloses an ultrasound image segmentation method, system, medium, and terminal based on portable devices, relating to the fields of artificial intelligence and medicine. Its main purpose is to improve the high latency of segmentation results caused by the mismatch between hardware computing power and model complexity. By sequentially solidifying the computational process and optimizing the hardware level of the ultrasound image segmentation model, the segmentation accuracy of the model is ensured while reducing the number of parameters and computational complexity. This allows the model to run on portable devices with limited hardware computing power, meeting the real-time interactive diagnostic needs in scenarios such as primary healthcare institutions or mobile screening.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and medical technology, and in particular to an ultrasound image segmentation method and system, medium, and terminal based on a portable device. Background Technology

[0002] With the continuous development of artificial intelligence, deep learning-based computer-aided diagnostic systems have been widely used in the field of ultrasound image segmentation. These systems automatically locate lesion regions and extract features to assist doctors in diagnosis, significantly improving diagnostic efficiency. The encoder-decoder architecture is the most mainstream approach. To meet the ever-increasing demand for segmentation accuracy, the optimization of segmentation models has primarily focused on improving segmentation precision, resulting in a huge number of parameters and high computational complexity.

[0003] However, when applied to primary healthcare institutions or mobile screening scenarios, these scenarios typically use low-cost, low-power portable devices with limited hardware computing power, which cannot support the complexity of the model. As a result, it is very easy to cause high latency in segmentation results or even system crashes, thus failing to meet the diagnostic needs of real-time interaction. Summary of the Invention

[0004] In view of this, this application provides an ultrasound image segmentation method, system, medium, and terminal based on a portable device, with the main purpose of improving the problem of high delay in segmentation results caused by the mismatch between hardware computing power and model complexity.

[0005] According to one aspect of this application, an ultrasound image segmentation method based on a portable device is provided, comprising:

[0006] Obtain the ultrasound image segmentation model that has completed model training, and perform a calculation process solidification process on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model;

[0007] Obtain the graph processing module parameters of the current portable device, perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters, and obtain the engine file corresponding to the ultrasound image segmentation model. The engine file is used to record the running scheme of the ultrasound image segmentation model that matches the current portable device.

[0008] The engine file is loaded and deployed into the current portable device, resulting in a fully deployed portable device.

[0009] Using the currently deployed portable device, the ultrasound image to be segmented is processed to obtain the lesion segmentation result image.

[0010] Preferably, the step of obtaining the image processing module parameters of the current portable device, and performing hardware-level optimization processing on the static ultrasound image segmentation model based on the image processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model includes:

[0011] Obtain the graph processing module parameters of the current portable device, and select the engine kernel function that matches the graph processing module parameters;

[0012] Referring to a predefined model layer fusion mode, the model layer sequences that can be fused are identified from the static ultrasound image segmentation model, and the model layer fusion processing is performed on the model layer sequences that can be fused to obtain the static ultrasound image segmentation model after model layer fusion processing. The predefined model layer fusion mode is used to record the model layer sequences that can be fused.

[0013] The model parameters of the static ultrasound image segmentation model are subjected to precision conversion processing to obtain the precision-converted model parameters;

[0014] Based on the engine kernel function, the static ultrasound image segmentation model after model layer fusion processing, and the model parameters after precision conversion processing, a model running scheme matching the current portable device is generated, and the model running scheme is serialized to obtain the engine file corresponding to the ultrasound image segmentation model.

[0015] Preferably, loading and deploying the engine file into the current portable device to obtain a fully deployed current portable device includes:

[0016] Load the engine file into the current portable device;

[0017] In the image processing module of the current portable device, an ultrasound image preprocessing thread, an ultrasound image segmentation thread, and an ultrasound image postprocessing thread are created respectively.

[0018] The ultrasound image preprocessing operator kernel function is deployed in the ultrasound image preprocessing thread to obtain the deployed ultrasound image preprocessing thread, so as to perform preprocessing on the ultrasound image in the deployed ultrasound image preprocessing thread.

[0019] The engine file is deserialized and loaded into the ultrasound image segmentation thread to obtain a fully deployed ultrasound image segmentation thread, which then performs segmentation processing on the ultrasound image.

[0020] The ultrasound image post-processing operator kernel function is deployed in the ultrasound image post-processing thread to obtain the deployed ultrasound image post-processing thread, so as to generate the lesion segmentation result image using the deployed ultrasound image post-processing thread.

[0021] Based on the completed deployment of the ultrasound image preprocessing thread, the completed deployment of the ultrasound image segmentation thread, and the completed deployment of the ultrasound image postprocessing thread, the current portable device that has been completed deployment is obtained.

[0022] Preferably, the step of using the deployed portable device to segment the ultrasound image to obtain a lesion segmentation result image includes:

[0023] Acquire the previous ultrasound image to be segmented, the current ultrasound image to be segmented, and the next ultrasound image to be segmented;

[0024] Simultaneously, the next ultrasound image to be segmented is preprocessed in the deployed ultrasound image preprocessing thread, the current ultrasound image to be segmented is segmented in the deployed ultrasound image segmentation thread, and the lesion segmentation result image corresponding to the previous ultrasound image to be segmented is generated using the deployed ultrasound image postprocessing thread, so as to perform a pipelined parallel ultrasound image segmentation process.

[0025] Preferably, the step of obtaining the trained ultrasound image segmentation model and performing a computational process solidification process on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model includes:

[0026] Obtain the ultrasound image segmentation model that has completed model training;

[0027] The ultrasound image segmentation model is converted to a static image format to obtain a static image format ultrasound image segmentation model.

[0028] The constant expressions contained in the ultrasound image segmentation model of the static image format are selected, and the constant expressions are calculated and solidified in the ultrasound image segmentation model to obtain the ultrasound image segmentation model after constant folding.

[0029] Referring to a predefined operator fusion mode, operator sequences that are allowed to be fused are identified from the constant-folded ultrasound image segmentation model, and operator fusion processing is performed on the operator sequences that are allowed to be fused to obtain an ultrasound image segmentation model after operator fusion processing. The predefined operator fusion mode is used to record the operator sequences that are allowed to be fused.

[0030] The ultrasound image segmentation model after operator fusion is pruned to eliminate redundant branch structures, resulting in a static ultrasound image segmentation model.

[0031] Preferably, before obtaining the ultrasound image segmentation model that has completed model training, the method further includes:

[0032] Obtain an ultrasound image dataset, which includes a combination of multiple sets of ultrasound images and corresponding standard lesion segmentation result images;

[0033] The ultrasound image dataset is augmented to obtain a training dataset;

[0034] A lightweight encoder-decoder architecture was constructed as the initial ultrasound image segmentation model;

[0035] Based on the initial ultrasound image segmentation model, each ultrasound image in the training dataset is segmented to obtain multiple predicted lesion segmentation result images.

[0036] A loss function is used to calculate the loss value between the multiple predicted lesion segmentation result images and the multiple standard lesion segmentation result images;

[0037] Based on the calculated loss value, the model parameters of the initial ultrasound image segmentation model are adjusted, and the initial ultrasound image segmentation model is trained and the loss value is calculated again until the convergence condition is met, thus obtaining the ultrasound image segmentation model that has completed model training.

[0038] Preferably, the construction of the lightweight encoder-decoder architecture as the initial ultrasound image segmentation model includes:

[0039] By sequentially connecting multiple reverse bottleneck modules and multiple tokenized multilayer sensing modules, a lightweight encoder is obtained.

[0040] Multiple tokenized multilayer perceptron modules and multiple convolutional modules are sequentially connected to obtain a lightweight decoder;

[0041] The lightweight encoder, the attention-gated dilated convolution module, and the lightweight decoder are connected sequentially to obtain the lightweight encoder-decoder architecture. The attention gating mechanism of the attention-gated dilated convolution module is used to capture features from two dimensions: channel information and spatial information.

[0042] According to another aspect of this application, an ultrasound image segmentation system based on a portable device is provided, comprising:

[0043] The model calculation process solidification module is used to obtain the ultrasound image segmentation model that has completed model training, and to solidify the calculation process of the ultrasound image segmentation model to obtain a static ultrasound image segmentation model.

[0044] The model hardware-level optimization module is used to obtain the graph processing module parameters of the current portable device, and perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model. The engine file is used to record the running scheme of the ultrasound image segmentation model that matches the current portable device.

[0045] The model deployment module is used to load the engine file and deploy it on the current portable device, resulting in a deployed current portable device.

[0046] The ultrasound image segmentation module is used to segment ultrasound images to be segmented using the currently deployed portable device to obtain lesion segmentation result images.

[0047] Preferably, the model hardware-level optimization module is used for:

[0048] Obtain the graph processing module parameters of the current portable device, and select the engine kernel function that matches the graph processing module parameters;

[0049] Referring to a predefined model layer fusion mode, the model layer sequences that can be fused are identified from the static ultrasound image segmentation model, and the model layer fusion processing is performed on the model layer sequences that can be fused to obtain the static ultrasound image segmentation model after model layer fusion processing. The predefined model layer fusion mode is used to record the model layer sequences that can be fused.

[0050] The model parameters of the static ultrasound image segmentation model are subjected to precision conversion processing to obtain the precision-converted model parameters;

[0051] Based on the engine kernel function, the static ultrasound image segmentation model after model layer fusion processing, and the model parameters after precision conversion processing, a model running scheme matching the current portable device is generated, and the model running scheme is serialized to obtain the engine file corresponding to the ultrasound image segmentation model.

[0052] Preferably, the model deployment module is used for:

[0053] Load the engine file into the current portable device;

[0054] In the image processing module of the current portable device, an ultrasound image preprocessing thread, an ultrasound image segmentation thread, and an ultrasound image postprocessing thread are created respectively.

[0055] The ultrasound image preprocessing operator kernel function is deployed in the ultrasound image preprocessing thread to obtain the deployed ultrasound image preprocessing thread, so as to perform preprocessing on the ultrasound image in the deployed ultrasound image preprocessing thread.

[0056] The engine file is deserialized and loaded into the ultrasound image segmentation thread to obtain a fully deployed ultrasound image segmentation thread, which then performs segmentation processing on the ultrasound image.

[0057] The ultrasound image post-processing operator kernel function is deployed in the ultrasound image post-processing thread to obtain the deployed ultrasound image post-processing thread, so as to generate the lesion segmentation result image using the deployed ultrasound image post-processing thread.

[0058] Based on the completed deployment of the ultrasound image preprocessing thread, the completed deployment of the ultrasound image segmentation thread, and the completed deployment of the ultrasound image postprocessing thread, the current portable device that has been completed deployment is obtained.

[0059] Preferably, the ultrasound image segmentation module is used for:

[0060] Acquire the previous ultrasound image to be segmented, the current ultrasound image to be segmented, and the next ultrasound image to be segmented;

[0061] Simultaneously, the next ultrasound image to be segmented is preprocessed in the deployed ultrasound image preprocessing thread, the current ultrasound image to be segmented is segmented in the deployed ultrasound image segmentation thread, and the lesion segmentation result image corresponding to the previous ultrasound image to be segmented is generated using the deployed ultrasound image postprocessing thread, so as to perform a pipelined parallel ultrasound image segmentation process.

[0062] Preferably, the model calculation process solidification module is used for:

[0063] Obtain the ultrasound image segmentation model that has completed model training;

[0064] The ultrasound image segmentation model is converted to a static image format to obtain a static image format ultrasound image segmentation model.

[0065] The constant expressions contained in the ultrasound image segmentation model of the static image format are selected, and the constant expressions are calculated and solidified in the ultrasound image segmentation model to obtain the ultrasound image segmentation model after constant folding.

[0066] Referring to a predefined operator fusion mode, operator sequences that are allowed to be fused are identified from the constant-folded ultrasound image segmentation model, and operator fusion processing is performed on the operator sequences that are allowed to be fused to obtain an ultrasound image segmentation model after operator fusion processing. The predefined operator fusion mode is used to record the operator sequences that are allowed to be fused.

[0067] The ultrasound image segmentation model after operator fusion is pruned to eliminate redundant branch structures, resulting in a static ultrasound image segmentation model.

[0068] Preferably, before the model calculation process solidification module, the system further includes a model building module, used for:

[0069] Obtain an ultrasound image dataset, which includes a combination of multiple sets of ultrasound images and corresponding standard lesion segmentation result images;

[0070] The ultrasound image dataset is augmented to obtain a training dataset;

[0071] A lightweight encoder-decoder architecture was constructed as the initial ultrasound image segmentation model;

[0072] Based on the initial ultrasound image segmentation model, each ultrasound image in the training dataset is segmented to obtain multiple predicted lesion segmentation result images.

[0073] A loss function is used to calculate the loss value between the multiple predicted lesion segmentation result images and the multiple standard lesion segmentation result images;

[0074] Based on the calculated loss value, the model parameters of the initial ultrasound image segmentation model are adjusted, and the initial ultrasound image segmentation model is trained and the loss value is calculated again until the convergence condition is met, thus obtaining the ultrasound image segmentation model that has completed model training.

[0075] Preferably, the model building module is further used for:

[0076] By sequentially connecting multiple reverse bottleneck modules and multiple tokenized multilayer sensing modules, a lightweight encoder is obtained.

[0077] Multiple tokenized multilayer perceptron modules and multiple convolutional modules are sequentially connected to obtain a lightweight decoder;

[0078] The lightweight encoder, the attention-gated dilated convolution module, and the lightweight decoder are connected sequentially to obtain the lightweight encoder-decoder architecture. The attention gating mechanism of the attention-gated dilated convolution module is used to capture features from two dimensions: channel information and spatial information.

[0079] According to another aspect of this application, a storage medium is provided that stores at least one executable instruction, which causes a processor to perform operations corresponding to the ultrasound image segmentation method based on a portable device described above.

[0080] According to another aspect of this application, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0081] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the ultrasound image segmentation method based on the portable device described above.

[0082] By employing the above technical solutions, the technical solutions provided in the embodiments of this application have at least the following advantages:

[0083] This application provides an ultrasound image segmentation method, system, medium, and terminal based on a portable device. First, a trained ultrasound image segmentation model is acquired, and its computational flow is solidified to obtain a static ultrasound image segmentation model. Second, the graph processing module parameters of the current portable device are acquired, and hardware-level optimization is performed on the static ultrasound image segmentation model based on these parameters to obtain an engine file corresponding to the ultrasound image segmentation model. The engine file records the running scheme of the ultrasound image segmentation model matching the current portable device. Third, the engine file is loaded and deployed on the current portable device to obtain a deployed portable device. Finally, the deployed portable device is used to segment the ultrasound image to be segmented, obtaining the lesion segmentation result image. Compared with existing technologies, this application's embodiment, by sequentially solidifying the computational flow and performing hardware-level optimization on the ultrasound image segmentation model, ensures the model's segmentation accuracy while reducing the number of model parameters and computational complexity. This allows the model to run on portable devices with limited hardware computing power, meeting the real-time interactive diagnostic needs in scenarios such as primary healthcare institutions or mobile screening.

[0084] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0085] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0086] Figure 1 A flowchart of an ultrasound image segmentation method based on a portable device provided in an embodiment of this application is shown;

[0087] Figure 2 A flowchart of another ultrasound image segmentation method based on a portable device provided in an embodiment of this application is shown;

[0088] Figure 3 The diagram illustrates a lightweight encoder-decoder architecture provided in an embodiment of this application.

[0089] Figure 4 A structural diagram of the attention gating mechanism provided in an embodiment of this application is shown;

[0090] Figure 5 The diagram shows the structure of the attention-gated dilated convolution module provided in an embodiment of this application.

[0091] Figure 6 This illustration shows a block diagram of an ultrasound image segmentation device based on a portable device, according to an embodiment of this application.

[0092] Figure 7 A schematic diagram of the structure of a terminal provided in an embodiment of this application is shown. Detailed Implementation

[0093] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0094] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0095] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0096] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0097] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0098] The embodiments of this application can be applied to computer systems / servers that can operate with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems, etc.

[0099] Computer systems / servers can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked through a communication network. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0100] This application provides an ultrasound image segmentation method based on a portable device, such as... Figure 1 As shown, the method includes:

[0101] 101. Obtain the ultrasound image segmentation model that has completed model training, and perform calculation process solidification processing on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model.

[0102] The ultrasound image segmentation model is built upon a lightweight encoder-decoder architecture, ensuring segmentation accuracy while further reducing the number of parameters and computational complexity. The computational process solidification process reduces the number of parameters and computational complexity at the model level, including static graph format conversion, constant folding, operator fusion, and pruning operations. The static ultrasound image segmentation model represents the static computational graph of the ultrasound image segmentation model. In this embodiment, the current execution end can be an optimization module of the ultrasound image segmentation model, used to optimize the model so that it can run on portable devices with limited hardware computing power, thereby meeting the real-time interactive diagnostic needs in primary healthcare institutions or mobile screening scenarios.

[0103] 102. Obtain the graph processing module parameters of the current portable device, and perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model.

[0104] The engine file records the execution plan of the ultrasound image segmentation model that matches the current portable device. The execution plan is the runtime execution plan, including but not limited to the execution order of operators, the selected optimal kernel function, the memory allocation scheme, the data flow transmission method, and hardware instructions. The current portable device can be a Jetson device. The graph processing module parameters are GPU parameters, such as GPU architecture. Hardware-level optimization is used to reduce the number of model parameters and computational complexity at the hardware level, including model layer fusion, precision conversion, serialization, and other operations.

[0105] 103. Load and deploy the engine file on the current portable device to obtain the current portable device after deployment.

[0106] It should be noted that, in order to improve the segmentation efficiency of ultrasound images, multiple thread groups can be configured in the current portable device. Each thread group contains three processing threads: a preprocessing thread, a segmentation thread, and a postprocessing thread. The ultrasound image segmentation task is divided into preprocessing, segmentation, and postprocessing stages. The preprocessing thread is responsible for executing the preprocessing stage tasks, the segmentation thread is responsible for executing the segmentation stage tasks, and the postprocessing thread is responsible for executing the postprocessing stage tasks. The tasks are executed in a pipelined parallel manner to improve segmentation efficiency.

[0107] 104. Using the currently deployed portable device, segment the ultrasound image to be segmented to obtain the lesion segmentation result image.

[0108] In this embodiment, a batch of ultrasound images to be segmented are input into the deployed portable device. First, the segmentation task of each ultrasound image to be segmented is divided into a preprocessing subtask, a segmentation subtask, and a postprocessing subtask. Then, each subtask is passed to the corresponding processing thread queue to wait for processing. During the processing, the ultrasound image preprocessing thread performs preprocessing on the next ultrasound image to be segmented, the ultrasound image segmentation thread performs segmentation on the current ultrasound image to be segmented, and the ultrasound image postprocessing thread generates the lesion segmentation result image corresponding to the previous ultrasound image to be segmented. Batch ultrasound image segmentation is performed in a pipelined parallel manner, thereby improving segmentation efficiency and ensuring timeliness.

[0109] Compared with the prior art, the embodiments of this application perform sequential computational process solidification and hardware-level optimization on the ultrasound image segmentation model, which not only ensures the segmentation accuracy of the model, but also reduces the number of model parameters and computational complexity, enabling the model to run on portable devices with limited hardware computing power, and meeting the real-time interactive diagnostic needs in scenarios such as primary healthcare institutions or mobile screening.

[0110] This application provides another method for ultrasound image segmentation based on a portable device, such as... Figure 2 As shown, the method includes:

[0111] 201. Construct a lightweight encoder-decoder architecture as the initial ultrasound image segmentation model.

[0112] Accordingly, step 201 of the embodiment specifically includes: sequentially connecting multiple reverse bottleneck modules and multiple tokenized multilayer perception modules to obtain a lightweight encoder; sequentially connecting multiple tokenized multilayer perception modules and multiple convolutional modules to obtain a lightweight decoder; and sequentially connecting the lightweight encoder, the attention-gated dilated convolutional module, and the lightweight decoder to obtain a lightweight encoder-decoder architecture, wherein the attention-gated dilated convolutional module's attention gating mechanism is used to capture features from both channel information and spatial information dimensions.

[0113] Specifically, lightweight encoder-decoder architecture, such as Figure 3 As shown in the figure, the left side represents a lightweight encoder, which consists of three inverse bottleneck modules and two tokenized multilayer perceptron (MLP) modules connected sequentially. This encoder is used to extract deep semantic features from ultrasound images layer by layer. The number of channels in each layer of the encoder can be set as follows: C1=16, C2=32, C3=128, C4=160, C5=256. The feature extraction process involves: converting the image... Input the first reverse bottleneck module ( This represents the image numbered 0. Represents the real number field. Indicates the height of the original input image. This represents the width of the original input image. Represent a A real matrix is ​​used to downsample the image with a stride of 2, resulting in the output feature map of the first layer of the encoder stage. (Represents the output feature map of layer 1) The space dimensions are (The number of channels is C1=16); further, Inputting the second inverse bottleneck module yields the output feature map of the second layer. Furthermore, Inputting the third inverse bottleneck module yields the output feature map of the third layer. Furthermore, Inputting the first tokenized multilayer perceptron module yields the output feature map of the fourth layer. Furthermore, Inputting the second tokenized multilayer perceptron module yields the output feature map of the 5th layer. The right side of the diagram shows a lightweight decoder, which can employ a symmetrical structure, consisting of two tokenized multilayer perceptron modules and three convolutional modules connected sequentially. The number of channels in each layer of the decoder can be set as follows: C6=384, C4=160, C3=128, C2=32, C1=16. The decoder processing procedure is as follows: the feature image passed through the attention-gated dilated convolutional module is input into the first tokenized multilayer perceptron module, upsampled to... The output feature map of the first layer of the decoder stage is obtained. Furthermore, The input is fed into the second tokenized multilayer sensing module and upsampled to... The output feature map of the second layer is obtained. Furthermore, The input is fed into the first convolutional module and upsampled to... The output feature map of the third layer is obtained. Furthermore, The input is fed into the second convolutional module and upsampled to... The output feature map of the 4th layer is obtained. Furthermore, The input is fed into the third convolutional module and upsampled to... The final lesion segmentation result image is obtained. Furthermore, due to the noise dispersion and shadowing effects in some ultrasound images, such as breast ultrasound images, large convolutional kernels can be used to better capture contextual information and avoid mistaking noise for salient features. However, large convolutional kernels increase the number of model parameters and are not suitable for portable devices. To improve performance without increasing the number of parameters, this embodiment adds an attention-gated dilated convolutional module (AGDT module) between the encoder and decoder. The attention-gated dilated convolutional module uses its attention gating mechanism to capture features from both channel and spatial dimensions, enhancing the expressive power of key regions and the ability to acquire multi-scale context. This not only improves the model's segmentation performance but also its segmentation efficiency. Specifically, the structure of the attention gating mechanism is as follows: Figure 4 As shown in the figure Let g represent the feature map corresponding to the Lth layer in the encoder, and g represent the feature map corresponding to the Lth layer in the decoder. The feature map of the next layer is the gated signal. Since a deeper g indicates that the model learns more knowledge, the information contained in g can serve as the focus of the model's subsequent learning. The spatial resolution of g is... Half of it, first upsample g to make the spatial resolution size equal to Equal, further, respectively for Perform a 5×5 depthwise convolution with g, extracting only the spatial information of the features in this step. This is similar to the first step of depthwise separable convolution. Further, g and... The convolutional features are concatenated, and 1×1 convolutions are used to extract channel information, thereby reducing the complexity of the attention gate and model parameters. Finally, Multiply by the calculated attention coefficient To overlay the information in g onto This allows attention to be directed to the target area. The attention gating mechanism can be expressed by the following formula.

[0114] ,

[0115] ,

[0116] ,

[0117] ,

[0118] ,

[0119] in, This represents the feature result obtained by applying depthwise convolution to the upsampled g. This represents a 5×5 depthwise convolution. This indicates that g is upsampled to double its spatial size for better matching. size, Indicates to The feature results after applying a 5×5 depthwise convolution, This represents the feature map corresponding to the Lth layer in the encoder. Indicates will and After concatenation, the features are processed by 1×1 convolution and ReLU activation function. Represents a 1×1 convolution. This represents the superposition of spatial dimensions. This represents the activation function. Indicates the attention coefficient. This represents the activation function. Indicates batch normalization. This represents the weighted encoder feature map, i.e., the output of the attention-gated module. Finally, the attention-gated mechanism is applied to the attention-gated dilated convolution module, the structure of which is as follows: Figure 5 As shown, the encoder output is used as the input of the attention-gated dilated convolution module, and then concatenated with a 3×3 convolution through an attention gating mechanism and a dilated convolution to obtain the output of the attention-gated dilated convolution module. Finally, the output is used as the input of the decoder.

[0120] It's worth noting that the use of a tokenized multilayer perceptron (MLP) in the encoder and decoder improves the model's segmentation efficiency. In the decoder, the last three layers utilize lightweight ordinary convolutional modules to reduce computational costs. The decoder's upsampling process employs bilinear interpolation, which reduces model parameters while preserving the detailed representation of the reconstructed image. Furthermore, to enhance the effectiveness of skip connections, an attention gating mechanism is introduced at the connection points to filter effective features.

[0121] 202. Train the initial ultrasound image segmentation model to obtain the trained ultrasound image segmentation model.

[0122] Accordingly, step 202 of the embodiment specifically includes: acquiring an ultrasound image dataset, wherein the ultrasound image dataset includes a combination of multiple sets of ultrasound images and corresponding standard lesion segmentation result images; performing data augmentation processing on the ultrasound image dataset to obtain a training dataset; constructing a lightweight encoder-decoder architecture as an initial ultrasound image segmentation model; based on the initial ultrasound image segmentation model, segmenting each ultrasound image in the training dataset to obtain multiple predicted lesion segmentation result images; using a loss function to calculate the loss value between the multiple predicted lesion segmentation result images and the multiple standard lesion segmentation result images; adjusting the model parameters of the initial ultrasound image segmentation model according to the calculated loss value, and continuing to train the initial ultrasound image segmentation model and calculate the loss value until the convergence condition is reached, thereby obtaining an ultrasound image segmentation model that has completed model training.

[0123] The ultrasound image dataset can utilize multi-source data, such as a combination of private and public datasets. Each ultrasound image must be accompanied by a corresponding standard lesion segmentation result image. Data augmentation processing can include operations such as cropping, translation, random horizontal mirroring, random vertical mirroring, scaling, random rotation, and Gaussian blurring.

[0124] 203. Obtain the ultrasound image segmentation model that has completed model training, and perform calculation process solidification processing on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model.

[0125] Accordingly, step 203 of the embodiment specifically includes: acquiring the ultrasound image segmentation model that has completed model training; converting the ultrasound image segmentation model to a static graph format to obtain a static graph format ultrasound image segmentation model; filtering out the constant expressions contained in the static graph format ultrasound image segmentation model, and performing calculation and solidification processing on the constant expressions in the ultrasound image segmentation model to obtain a constant-folded ultrasound image segmentation model; identifying the operator sequences that can be fused from the constant-folded ultrasound image segmentation model according to a predefined operator fusion mode, and performing operator fusion processing on the operator sequences that can be fused to obtain an operator-fused ultrasound image segmentation model, wherein the predefined operator fusion mode is used to record the operator sequences that can be fused; and pruning the operator-fused ultrasound image segmentation model to eliminate redundant branch structures to obtain a static ultrasound image segmentation model.

[0126] The static graph format conversion process converts the ultrasound image segmentation model into a static graph format, such as ONNX, which is an intermediate representation level static computation graph. This format explicitly records the operator execution order, tensor dimensions, and computational dependencies, thus solidifying the computational flow and avoiding the computational power consumed by rebuilding the computation graph each time inference occurs. Furthermore, the ONNX format can be loaded by high-performance inference engines such as TensorRT, allowing graph structure analysis to be completed during the loading phase. During the inference phase, only a fixed graph topology needs to be executed, effectively avoiding the parsing overhead caused by dynamic computation graphs. The constant folding process pre-compiles the constant representation subgraphs contained in the static graph format ultrasound image segmentation model, such as shape calculations and multiply-accumulate constants, during the compilation phase to avoid repeated execution during inference. The predefined operator fusion mode records the operator sequences that can be fused, such as Conv+BN+ReLU, and after identifying the operator sequences that can be fused, they are fused into a subgraph node, reducing intermediate storage and accelerating memory bandwidth access. The pruning process removes unused branches or temporary intermediate tensors in the model structure to reduce operator redundancy. In addition, it can identify and simplify identity operations such as Mul(1), Add(0), Div(1), and replace them with pass-through operators or delete them.

[0127] It should be noted that by solidifying the computation process, the static execution path of the model can be simplified, thereby effectively compressing the size of the computation graph.

[0128] 204. Obtain the graph processing module parameters of the current portable device, and perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model.

[0129] Accordingly, step 204 of the embodiment specifically includes: obtaining the graph processing module parameters of the current portable device, selecting an engine kernel function that matches the graph processing module parameters; identifying the model layer sequence that can be fused from the static ultrasound image segmentation model by referring to a predefined model layer fusion mode, and performing model layer fusion processing on the model layer sequence that can be fused to obtain a static ultrasound image segmentation model after model layer fusion processing, wherein the predefined model layer fusion mode is used to record the model layer sequence that can be fused; performing precision conversion processing on the model parameters of the static ultrasound image segmentation model to obtain the model parameters after precision conversion processing; generating a model running scheme that matches the current portable device based on the engine kernel function, the static ultrasound image segmentation model after model layer fusion processing, and the model parameters after precision conversion processing, and performing serialization processing on the model running scheme to obtain the engine file corresponding to the ultrasound image segmentation model.

[0130] The process involves several key components: First, an engine kernel function matching the graph processing module parameters is selected. This involves choosing the fastest kernel function from a pool of pre-generated kernel functions that executes on the current portable device's graph processing module. Second, a predefined model layer fusion mode records allowed model layer sequences, such as a continuous sequence of convolutional layers, batch normalization layers, and activation function layers. After identifying these sequences, they are fused into a single kernel function to simplify kernel function calls, reduce memory read / write and kernel scheduling overhead, and free up computing power. Third, precision conversion performs half-precision calculations on non-critical paths (such as intermediate feature extraction layers) while maintaining segmentation accuracy, reducing model size. Fourth, a model execution plan, including but not limited to the execution order of operators, the selected optimal kernel function, memory allocation scheme, data flow transmission method, and hardware instructions, is used. Fifth, an engine file records the serialized model execution plan, enabling direct ultrasound image segmentation during deployment by simply deserializing and loading the engine file, eliminating the need for time-consuming model loading and optimization processes and further ensuring rapid model startup and efficient operation.

[0131] 205. Load and deploy the engine file on the current portable device to obtain the current portable device after deployment.

[0132] Accordingly, step 205 of the embodiment specifically includes: loading the engine file into the current portable device; creating an ultrasound image preprocessing thread, an ultrasound image segmentation thread, and an ultrasound image postprocessing thread in the image processing module of the current portable device; deploying the ultrasound image preprocessing operator kernel function in the ultrasound image preprocessing thread to obtain a deployed ultrasound image preprocessing thread, so as to preprocess the ultrasound image in the deployed ultrasound image preprocessing thread; deserializing and loading the engine file into the ultrasound image segmentation thread to obtain a deployed ultrasound image segmentation thread, so as to segment the ultrasound image in the deployed ultrasound image segmentation thread; deploying the ultrasound image postprocessing operator kernel function in the ultrasound image postprocessing thread to obtain a deployed ultrasound image postprocessing thread, so as to generate a lesion segmentation result image using the deployed ultrasound image postprocessing thread; and obtaining the deployed current portable device based on the deployed ultrasound image preprocessing thread, the deployed ultrasound image segmentation thread, and the deployed ultrasound image postprocessing thread.

[0133] 206. Using the currently deployed portable device, the ultrasound image to be segmented is processed to obtain the lesion segmentation result image.

[0134] Accordingly, step 206 of the embodiment specifically includes: acquiring the previous ultrasound image to be segmented, the current ultrasound image to be segmented, and the next ultrasound image to be segmented; simultaneously preprocessing the next ultrasound image to be segmented in the deployed ultrasound image preprocessing thread, segmenting the current ultrasound image to be segmented in the deployed ultrasound image segmentation thread, and generating the lesion segmentation result image corresponding to the previous ultrasound image to be segmented using the deployed ultrasound image postprocessing thread, so as to perform a pipelined parallel ultrasound image segmentation process.

[0135] Specifically, based on the asynchronous parallel execution mechanism of the TensorRT Plugin interface, the three stages of preprocessing, segmentation, and postprocessing can be divided into independent CUDA streams, and the execution flow of TensorRT can be extended and scheduled through the Plugin mechanism to achieve pipelined parallel processing.

[0136] This application provides an ultrasound image segmentation method based on a portable device. First, a trained ultrasound image segmentation model is acquired, and its computational flow is solidified to obtain a static ultrasound image segmentation model. Second, the graph processing module parameters of the current portable device are acquired, and hardware-level optimization is performed on the static ultrasound image segmentation model based on these parameters to obtain an engine file corresponding to the ultrasound image segmentation model. The engine file records the running scheme of the ultrasound image segmentation model matching the current portable device. Third, the engine file is loaded and deployed on the current portable device to obtain a deployed portable device. Finally, the deployed portable device is used to segment the ultrasound image to be segmented, obtaining the lesion segmentation result image. Compared with existing technologies, this application embodiment, by sequentially solidifying the computational flow and performing hardware-level optimization on the ultrasound image segmentation model, ensures the model's segmentation accuracy while reducing the number of model parameters and computational complexity. This allows the model to run on portable devices with limited hardware computing power, meeting the real-time interactive diagnostic needs in scenarios such as primary healthcare institutions or mobile screening.

[0137] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this application provides an ultrasound image segmentation system based on a portable device, such as... Figure 6 As shown, the system includes:

[0138] Model calculation process solidification module 31, model hardware level optimization module 32, model deployment module 33, ultrasound image segmentation module 34;

[0139] The model calculation process solidification module 31 is used to obtain the ultrasound image segmentation model that has completed model training, and to perform calculation process solidification processing on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model.

[0140] The model hardware-level optimization module 32 is used to obtain the graph processing module parameters of the current portable device, and perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model. The engine file is used to record the running scheme of the ultrasound image segmentation model that matches the current portable device.

[0141] The model deployment module 33 is used to load the engine file and deploy it in the current portable device to obtain a current portable device with complete deployment;

[0142] The ultrasound image segmentation module 34 is used to segment the ultrasound image to be segmented using the currently deployed portable device to obtain the lesion segmentation result image.

[0143] In specific application scenarios, the model hardware-level optimization module is used for:

[0144] Obtain the graph processing module parameters of the current portable device, and select the engine kernel function that matches the graph processing module parameters;

[0145] Referring to a predefined model layer fusion mode, the model layer sequences that can be fused are identified from the static ultrasound image segmentation model, and the model layer fusion processing is performed on the model layer sequences that can be fused to obtain the static ultrasound image segmentation model after model layer fusion processing. The predefined model layer fusion mode is used to record the model layer sequences that can be fused.

[0146] The model parameters of the static ultrasound image segmentation model are subjected to precision conversion processing to obtain the precision-converted model parameters;

[0147] Based on the engine kernel function, the static ultrasound image segmentation model after model layer fusion processing, and the model parameters after precision conversion processing, a model running scheme matching the current portable device is generated, and the model running scheme is serialized to obtain the engine file corresponding to the ultrasound image segmentation model.

[0148] In specific application scenarios, the model deployment module is used for:

[0149] Load the engine file into the current portable device;

[0150] In the image processing module of the current portable device, an ultrasound image preprocessing thread, an ultrasound image segmentation thread, and an ultrasound image postprocessing thread are created respectively.

[0151] The ultrasound image preprocessing operator kernel function is deployed in the ultrasound image preprocessing thread to obtain the deployed ultrasound image preprocessing thread, so as to perform preprocessing on the ultrasound image in the deployed ultrasound image preprocessing thread.

[0152] The engine file is deserialized and loaded into the ultrasound image segmentation thread to obtain a fully deployed ultrasound image segmentation thread, which then performs segmentation processing on the ultrasound image.

[0153] The ultrasound image post-processing operator kernel function is deployed in the ultrasound image post-processing thread to obtain the deployed ultrasound image post-processing thread, so as to generate the lesion segmentation result image using the deployed ultrasound image post-processing thread.

[0154] Based on the completed deployment of the ultrasound image preprocessing thread, the completed deployment of the ultrasound image segmentation thread, and the completed deployment of the ultrasound image postprocessing thread, the current portable device that has been completed deployment is obtained.

[0155] In specific application scenarios, the ultrasound image segmentation module is used for:

[0156] Acquire the previous ultrasound image to be segmented, the current ultrasound image to be segmented, and the next ultrasound image to be segmented;

[0157] Simultaneously, the next ultrasound image to be segmented is preprocessed in the deployed ultrasound image preprocessing thread, the current ultrasound image to be segmented is segmented in the deployed ultrasound image segmentation thread, and the lesion segmentation result image corresponding to the previous ultrasound image to be segmented is generated using the deployed ultrasound image postprocessing thread, so as to perform a pipelined parallel ultrasound image segmentation process.

[0158] In specific application scenarios, the model calculation process solidification module is used for:

[0159] Obtain the ultrasound image segmentation model that has completed model training;

[0160] The ultrasound image segmentation model is converted to a static image format to obtain a static image format ultrasound image segmentation model.

[0161] The constant expressions contained in the ultrasound image segmentation model of the static image format are selected, and the constant expressions are calculated and solidified in the ultrasound image segmentation model to obtain the ultrasound image segmentation model after constant folding.

[0162] Referring to a predefined operator fusion mode, operator sequences that are allowed to be fused are identified from the constant-folded ultrasound image segmentation model, and operator fusion processing is performed on the operator sequences that are allowed to be fused to obtain an ultrasound image segmentation model after operator fusion processing. The predefined operator fusion mode is used to record the operator sequences that are allowed to be fused.

[0163] The ultrasound image segmentation model after operator fusion is pruned to eliminate redundant branch structures, resulting in a static ultrasound image segmentation model.

[0164] In specific application scenarios, prior to the model calculation process solidification module, the system further includes a model building module, used for:

[0165] Obtain an ultrasound image dataset, which includes a combination of multiple sets of ultrasound images and corresponding standard lesion segmentation result images;

[0166] The ultrasound image dataset is augmented to obtain a training dataset;

[0167] A lightweight encoder-decoder architecture was constructed as the initial ultrasound image segmentation model;

[0168] Based on the initial ultrasound image segmentation model, each ultrasound image in the training dataset is segmented to obtain multiple predicted lesion segmentation result images.

[0169] A loss function is used to calculate the loss value between the multiple predicted lesion segmentation result images and the multiple standard lesion segmentation result images;

[0170] Based on the calculated loss value, the model parameters of the initial ultrasound image segmentation model are adjusted, and the initial ultrasound image segmentation model is trained and the loss value is calculated again until the convergence condition is met, thus obtaining the ultrasound image segmentation model that has completed model training.

[0171] In specific application scenarios, the model building module is also used for:

[0172] By sequentially connecting multiple reverse bottleneck modules and multiple tokenized multilayer sensing modules, a lightweight encoder is obtained.

[0173] Multiple tokenized multilayer perceptron modules and multiple convolutional modules are sequentially connected to obtain a lightweight decoder;

[0174] The lightweight encoder, the attention-gated dilated convolution module, and the lightweight decoder are connected sequentially to obtain the lightweight encoder-decoder architecture. The attention gating mechanism of the attention-gated dilated convolution module is used to capture features from two dimensions: channel information and spatial information.

[0175] This application provides an ultrasound image segmentation system based on a portable device. First, a trained ultrasound image segmentation model is acquired, and its computational flow is solidified to obtain a static ultrasound image segmentation model. Second, the graph processing module parameters of the current portable device are acquired, and hardware-level optimization is performed on the static ultrasound image segmentation model based on these parameters to obtain an engine file corresponding to the ultrasound image segmentation model. The engine file records the running scheme of the ultrasound image segmentation model matching the current portable device. Third, the engine file is loaded and deployed on the current portable device to obtain a deployed portable device. Finally, the deployed portable device is used to segment the ultrasound image to be segmented, obtaining the lesion segmentation result image. Compared with existing technologies, this application's embodiment, by sequentially solidifying the computational flow and performing hardware-level optimization on the ultrasound image segmentation model, ensures the model's segmentation accuracy while reducing the number of model parameters and computational complexity. This allows the model to run on portable devices with limited hardware computing power, meeting the real-time interactive diagnostic needs in scenarios such as primary healthcare institutions or mobile screening.

[0176] According to one embodiment of this application, a storage medium is provided, the storage medium storing at least one executable instruction that can execute the ultrasonic image segmentation method based on a portable device in any of the above method embodiments.

[0177] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0178] Figure 7 The diagram shows a structural schematic of a terminal according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the terminal.

[0179] like Figure 7 As shown, the terminal may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0180] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0181] Communication interface 404 is used to communicate with other network elements such as clients or other servers.

[0182] The processor 402 is used to execute program 410, specifically to perform the relevant steps in the above-described embodiment of the ultrasonic image segmentation method based on a portable device.

[0183] Specifically, program 410 may include program code that includes computer operation instructions.

[0184] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0185] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0186] Specifically, program 410 can be used to cause processor 402 to perform the following operations:

[0187] Obtain the ultrasound image segmentation model that has completed model training, and perform a calculation process solidification process on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model;

[0188] Obtain the graph processing module parameters of the current portable device, perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters, and obtain the engine file corresponding to the ultrasound image segmentation model. The engine file is used to record the running scheme of the ultrasound image segmentation model that matches the current portable device.

[0189] The engine file is loaded and deployed into the current portable device, resulting in a fully deployed portable device.

[0190] Using the currently deployed portable device, the ultrasound image to be segmented is processed to obtain the lesion segmentation result image.

[0191] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device for the aforementioned ultrasound image segmentation method based on portable devices, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0192] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0193] The methods and systems of this application may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this application are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this application may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this application. Thus, this application also covers recording media storing programs for performing the methods according to this application.

[0194] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0195] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for segmenting ultrasound images based on a portable device, characterized in that, include: Obtain the ultrasound image segmentation model that has completed model training, and perform a calculation process solidification process on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model; Obtain the graph processing module parameters of the current portable device, perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters, and obtain the engine file corresponding to the ultrasound image segmentation model. The engine file is used to record the running scheme of the ultrasound image segmentation model that matches the current portable device. The engine file is loaded and deployed into the current portable device, resulting in a fully deployed portable device. Using the currently deployed portable device, the ultrasound image to be segmented is processed to obtain the lesion segmentation result image; The process of obtaining the graph processing module parameters of the current portable device, and performing hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model includes: Obtain the graph processing module parameters of the current portable device, and select the engine kernel function that matches the graph processing module parameters; Referring to a predefined model layer fusion mode, the model layer sequences that can be fused are identified from the static ultrasound image segmentation model, and the model layer fusion processing is performed on the model layer sequences that can be fused to obtain the static ultrasound image segmentation model after model layer fusion processing. The predefined model layer fusion mode is used to record the model layer sequences that can be fused. The model parameters of the static ultrasound image segmentation model are subjected to precision conversion processing to obtain the precision-converted model parameters; Based on the engine kernel function, the static ultrasound image segmentation model after model layer fusion processing, and the model parameters after precision conversion processing, a model running scheme matching the current portable device is generated, and the model running scheme is serialized to obtain the engine file corresponding to the ultrasound image segmentation model.

2. The method according to claim 1, characterized in that, The step of loading and deploying the engine file into the current portable device to obtain a fully deployed current portable device includes: Load the engine file into the current portable device; In the image processing module of the current portable device, an ultrasound image preprocessing thread, an ultrasound image segmentation thread, and an ultrasound image postprocessing thread are created respectively. The ultrasound image preprocessing operator kernel function is deployed in the ultrasound image preprocessing thread to obtain the deployed ultrasound image preprocessing thread, so as to perform preprocessing on the ultrasound image in the deployed ultrasound image preprocessing thread. The engine file is deserialized and loaded into the ultrasound image segmentation thread to obtain a fully deployed ultrasound image segmentation thread, which then performs segmentation processing on the ultrasound image. The ultrasound image post-processing operator kernel function is deployed in the ultrasound image post-processing thread to obtain the deployed ultrasound image post-processing thread, so as to generate the lesion segmentation result image using the deployed ultrasound image post-processing thread. Based on the completed deployment of the ultrasound image preprocessing thread, the completed deployment of the ultrasound image segmentation thread, and the completed deployment of the ultrasound image postprocessing thread, the current portable device that has been completed deployment is obtained.

3. The method according to claim 1, characterized in that, The process of segmenting the ultrasound image to be segmented using the deployed portable device to obtain the lesion segmentation result image includes: Acquire the previous ultrasound image to be segmented, the current ultrasound image to be segmented, and the next ultrasound image to be segmented; Simultaneously, the next ultrasound image to be segmented is preprocessed in the deployed ultrasound image preprocessing thread, the current ultrasound image to be segmented is segmented in the deployed ultrasound image segmentation thread, and the lesion segmentation result image corresponding to the previous ultrasound image to be segmented is generated using the deployed ultrasound image postprocessing thread, so as to perform a pipelined parallel ultrasound image segmentation process.

4. The method according to claim 1, characterized in that, The process of acquiring the trained ultrasound image segmentation model and then performing a computational flow fixation process on the ultrasound image segmentation model to obtain a static ultrasound image segmentation model includes: Obtain the ultrasound image segmentation model that has completed model training; The ultrasound image segmentation model is converted to a static image format to obtain a static image format ultrasound image segmentation model. The constant expressions contained in the ultrasound image segmentation model of the static image format are selected, and the constant expressions are calculated and solidified in the ultrasound image segmentation model to obtain the ultrasound image segmentation model after constant folding. Referring to a predefined operator fusion mode, operator sequences that are allowed to be fused are identified from the constant-folded ultrasound image segmentation model, and operator fusion processing is performed on the operator sequences that are allowed to be fused to obtain an ultrasound image segmentation model after operator fusion processing. The predefined operator fusion mode is used to record the operator sequences that are allowed to be fused. The ultrasound image segmentation model after operator fusion is pruned to eliminate redundant branch structures, resulting in a static ultrasound image segmentation model.

5. The method according to claim 1, characterized in that, Before obtaining the ultrasound image segmentation model that has completed model training, the method further includes: Obtain an ultrasound image dataset, which includes a combination of multiple sets of ultrasound images and corresponding standard lesion segmentation result images; The ultrasound image dataset is augmented to obtain a training dataset; A lightweight encoder-decoder architecture was constructed as the initial ultrasound image segmentation model; Based on the initial ultrasound image segmentation model, each ultrasound image in the training dataset is segmented to obtain multiple predicted lesion segmentation result images. A loss function is used to calculate the loss value between multiple predicted lesion segmentation result images and multiple standard lesion segmentation result images; Based on the calculated loss value, the model parameters of the initial ultrasound image segmentation model are adjusted, and the initial ultrasound image segmentation model is trained and the loss value is calculated again until the convergence condition is met, thus obtaining the ultrasound image segmentation model that has completed model training.

6. The method according to claim 5, characterized in that, The construction of a lightweight encoder-decoder architecture as an initial ultrasound image segmentation model includes: By sequentially connecting multiple reverse bottleneck modules and multiple tokenized multilayer sensing modules, a lightweight encoder is obtained. Multiple tokenized multilayer perceptron modules and multiple convolutional modules are sequentially connected to obtain a lightweight decoder; The lightweight encoder, the attention-gated dilated convolution module, and the lightweight decoder are connected sequentially to obtain the lightweight encoder-decoder architecture. The attention gating mechanism of the attention-gated dilated convolution module is used to capture features from two dimensions: channel information and spatial information.

7. An ultrasound image segmentation system based on a portable device, characterized in that, include: The model calculation process solidification module is used to obtain the ultrasound image segmentation model that has completed model training, and to solidify the calculation process of the ultrasound image segmentation model to obtain a static ultrasound image segmentation model. The model hardware-level optimization module is used to obtain the graph processing module parameters of the current portable device, and perform hardware-level optimization processing on the static ultrasound image segmentation model based on the graph processing module parameters to obtain the engine file corresponding to the ultrasound image segmentation model. The engine file is used to record the running scheme of the ultrasound image segmentation model that matches the current portable device. The model deployment module is used to load the engine file and deploy it on the current portable device, resulting in a deployed current portable device. The ultrasound image segmentation module is used to segment the ultrasound image to be segmented using the currently deployed portable device to obtain the lesion segmentation result image; The model hardware-level optimization module is used for: Obtain the graph processing module parameters of the current portable device, and select the engine kernel function that matches the graph processing module parameters; Referring to a predefined model layer fusion mode, the model layer sequences that can be fused are identified from the static ultrasound image segmentation model, and the model layer fusion processing is performed on the model layer sequences that can be fused to obtain the static ultrasound image segmentation model after model layer fusion processing. The predefined model layer fusion mode is used to record the model layer sequences that can be fused. The model parameters of the static ultrasound image segmentation model are subjected to precision conversion processing to obtain the precision-converted model parameters; Based on the engine kernel function, the static ultrasound image segmentation model after model layer fusion processing, and the model parameters after precision conversion processing, a model running scheme matching the current portable device is generated, and the model running scheme is serialized to obtain the engine file corresponding to the ultrasound image segmentation model.

8. A storage medium storing at least one executable instruction, characterized in that, The executable instructions cause the processor to perform the operations corresponding to the ultrasound image segmentation method based on a portable device as described in any one of claims 1-6.

9. A terminal, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, characterized in that the executable instruction causes the processor to perform the operation corresponding to the ultrasound image segmentation method based on a portable device as described in any one of claims 1-6.

Images