Method for generating image segmentation model, and method and device for image segmentation

Abstract

The invention provides a method for generating an image segmentation model, and an image segmentation method and device. Wherein an image segmentation model comprises convolution operators deployed on calculation nodes respectively, and the method comprises the steps that an image sample set is acquired, and each image sample is marked with a segmentation mask; dividing each image sample into a plurality of image blocks, and distributing each image block to a corresponding computing node; training an image segmentation model on the computational nodes by using the image samples and the segmentation masks; wherein feature extraction processing involved in the training process comprises the steps that image information and boundary information of the distributed image blocks are processed in parallel through convolution operators on all computational nodes, local features corresponding to all the image blocks are obtained, and the boundary information comes from the computational nodes of the distributed adjacent image blocks. The training time can be shortened, and the training efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for generating an image segmentation model, and a method and device for image segmentation. Background Art

[0002] With the rapid development of deep learning technology, medical image segmentation based on deep learning has also been widely used in the medical field; medical image segmentation refers to the process of accurately dividing and marking different structures or regions in medical images. This process aims to identify and separate the anatomical structures, organs, tissues or lesions of interest in the image for further analysis, diagnosis and treatment planning.

[0003] Image segmentation models in related technologies, such as SAM (Segment Anything Model), can achieve accurate image segmentation, but their training process requires processing more than 11 million images and 1 billion masks, resulting in long training time and low training efficiency. Summary of the invention

[0004] In view of this, the embodiments of the present application are committed to providing a method for generating an image segmentation model, a method and apparatus for image segmentation, so as to shorten the training time and improve the training efficiency.

[0005] In a first aspect, a method for generating an image segmentation model is provided, wherein the image segmentation model of the device includes convolution operators respectively deployed on each computing node, and the method includes: obtaining an image sample set, each image sample being marked with a segmentation mask; dividing each image sample into multiple image blocks, and respectively assigning each image block to a corresponding computing node; using the image sample set and the segmentation mask, training the image segmentation model on the computing node; wherein the feature extraction processing involved in the training process includes: processing the image information and boundary information of the assigned image blocks in parallel through the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing node assigned adjacent image blocks.

[0006] In a second aspect, a method for image segmentation is provided, the method comprising: obtaining an image to be segmented; dividing the image to be segmented into multiple image blocks, and assigning each image block to a corresponding computing node respectively; performing segmentation processing on the multiple image blocks using an image segmentation model to obtain a segmentation result for the image to be segmented; wherein the image segmentation model comprises convolution operators respectively deployed on each computing node, and the feature extraction processing involved in the segmentation processing comprises: processing the image information and boundary information of the assigned image blocks in parallel through the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes assigned adjacent image blocks.

[0007] In a third aspect, a device for generating an image segmentation model is provided, wherein the image segmentation model of the device includes convolution operators respectively deployed on each computing node, and the device includes: a sample acquisition module, used to acquire an image sample set, each image sample is marked with a segmentation mask; an image processing module, used to divide each image sample into multiple image blocks, and respectively assign each image block to a corresponding computing node; a model training module, used to train the image segmentation model on the computing node using the image sample set and the segmentation mask; wherein the feature extraction processing involved in the training process includes: parallel processing of image information and boundary information of the assigned image blocks by the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing node assigned adjacent image blocks.

[0008] In a fourth aspect, a device for image segmentation is provided, which includes: an image acquisition module for acquiring an image to be segmented; an image processing module for dividing the image to be segmented into multiple image blocks, and assigning each image block to a corresponding computing node; an image segmentation module for performing segmentation processing on the multiple image blocks using an image segmentation model to obtain a segmentation result for the image to be segmented; wherein the image segmentation model includes convolution operators deployed on each computing node, respectively, and the feature extraction processing involved in the segmentation processing includes: parallel processing of image information and boundary information of the assigned image blocks by the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes to which adjacent image blocks are assigned.

[0009] According to a fifth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, and when the program instructions are read and executed by the one or more processors, the method described in the first aspect or the second aspect is executed.

[0010] According to a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0011] According to a seventh aspect, a program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to implement the method described in the first aspect or the second aspect.

[0012] The present application first distributes the multiple image blocks into which each image sample is divided to the corresponding computing nodes; next, the image information and boundary information of the distributed image blocks are processed in parallel by the convolution operators on each computing node to obtain the local features corresponding to each image block; next, the local features and segmentation masks corresponding to each image block are used to implement parallel training of the image segmentation model on each computing node, which can shorten the training time and improve the training efficiency compared with the SAM in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the architecture of the method for generating an image segmentation model or an application scenario of the image segmentation method provided in an embodiment of the present application;

[0014] Figure 2 is a flowchart of a method for generating an image segmentation model provided in an embodiment of the present application;

[0015] Figure 3 It is a structural schematic diagram of the image segmentation model provided in the embodiment of the present application;

[0016] Figure 4 is a schematic diagram of transmission boundary information provided by an embodiment of the present application;

[0017] Figure 5 is a flow chart of the image segmentation method provided in an embodiment of the present application;

[0018] Figure 6 is a schematic diagram of a device for generating an image segmentation model provided in an embodiment of the present application;

[0019] Figure 7 is a schematic diagram of an image segmentation device provided in an embodiment of the present application;

[0020] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] In this embodiment, the method for generating an image segmentation model may include: first, assigning multiple image blocks into which each image sample is divided to corresponding computing nodes; next, processing the image information and boundary information of the assigned image blocks in parallel through the convolution operators on each computing node to obtain local features corresponding to each image block; next, using the local features and segmentation masks corresponding to each image block to implement parallel training of the image segmentation model on each computing node, which can shorten the training time and improve the training efficiency compared to the SAM in the related art.

[0023] For ease of understanding, this article first introduces the application scenarios applicable to the method for generating an image segmentation model or the method for image segmentation provided by the present application, see Figure 1 , the distributed system includes a master node 101 and multiple computing nodes; wherein the master node 101 is connected to multiple computing nodes 102 in communication, for example, PCIe (Peripheral Component Interconnect Express), NVLink (connection), etc. NVLink is a bus communication protocol developed and launched by NVIDIA; multiple computing nodes 102 are connected in communication, for example, Message Queue (message queue), RPC (Remote Procedure Call, remote procedure call), DSM (Distributed Shared Memory, distributed shared memory), Socket (network socket), etc. wherein the master node 101 is used to manage multiple computing nodes 102 to train image segmentation models in parallel, and the master node 101 performs the following operations:

[0024] (1) Obtain a set of image samples, each of which is marked with a segmentation mask;

[0025] (2) Divide each image sample into multiple image blocks, and assign each image block to a corresponding computing node;

[0026] (3) Using each image sample and segmentation mask, the image segmentation model is trained on the computing node; wherein the feature extraction processing involved in the training process includes: using the convolution operator on each computing node to parallelly process the image information and boundary information of the assigned image block to obtain the local features corresponding to each image block, and the boundary information comes from the computing node assigned the adjacent image block.

[0027] Each node in the distributed system (i.e., the master node 101 or each computing node in the plurality of computing nodes 102) can be set as an electronic device, such as an independent server, or can be set in a server group, or can be set in a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in a cloud computing service system to solve the problems of difficult management and weak service scalability in traditional physical hosts and virtual private servers (VPS). Figure 1 In addition to the architecture shown, each node can also be set on a computer terminal with strong computing power.

[0028] Figure 2 A flowchart of a method for generating an image segmentation model provided in an embodiment of the present application, wherein the scheme is applied to Figure 1 The following is an example of the master node in Figure 1. Figure 2 As shown in , the image segmentation model includes a convolution operator deployed on each computing node, and the method may include the following steps:

[0029] Step S201: Obtain an image sample set, each image sample being marked with a segmentation mask.

[0030] Step S202: Divide each image sample into a plurality of image blocks, and assign each image block to a corresponding computing node.

[0031] Step S203: Using the image sample set and the segmentation mask, the image segmentation model is trained on the computing node; wherein the feature extraction processing involved in the training process includes: using the convolution operator on each computing node to parallelly process the image information and boundary information of the assigned image blocks to obtain the local features corresponding to each image block, and the boundary information comes from the computing node assigned the adjacent image blocks.

[0032] It can be seen that in the process of training the image segmentation model, each image block can be processed in parallel through the convolution operator on each computing node, which can shorten the training time and improve the training efficiency.

[0033] The following describes in detail each step in the above process and the effects that can be further produced in conjunction with the embodiments.

[0034] First, "the image segmentation model includes a convolution operator deployed on each computing node" is described in detail in conjunction with the embodiments.

[0035] In an embodiment of the present application, the image segmentation model includes a deep learning operator, which performs operations on input data through mathematical functions to extract features, establish patterns or make decisions. Deep learning operators include but are not limited to convolution operators, pooling operators, activation function operators, etc., which together constitute the core of the image segmentation model, enabling the model to handle complex data representation and learning tasks. For example, the convolution operator is used to identify the local features of each image block in each image sample, while activation functions such as ReLU (Rectified Linear Unit) introduce nonlinearity, enabling the image segmentation model to learn function mapping. The choice of deep learning operators affects the performance, efficiency and final learning results of the image segmentation model.

[0036] Here, the convolution operator convolves the image information and boundary information of each image block through a filter or convolution kernel. It can capture the local features of each image block in each image sample, providing key visual features for subsequent understanding and processing of image sample sets.

[0037] Before training the image segmentation model, the convolution operators included in the image segmentation model are pre-deployed on multiple computing nodes.

[0038] In one example, an image segmentation model is used to segment a three-dimensional image, such as a three-dimensional medical image.

[0039] Correspondingly, in this example, the above convolution operator is a three-dimensional convolution operator (such as Figure 3 The three-dimensional convolution operator (Sliceconv) in .

[0040] Continuing from the above, the image segmentation model in the embodiment of the present application includes an encoder, a decoder and a head, and the encoder and the decoder respectively include convolution operators deployed on each computing node.

[0041] exist Figure 3 In the present invention, the three-dimensional image segmentation model includes an encoder, a decoder and a head (output); wherein the encoder may include a three-dimensional convolution operator (Sliceconv) and a pooling layer (POOL), and the encoder is used to extract the feature representation of each image sample; the decoder includes upsampling (UPsample) and a three-dimensional convolution operator (Sliceconv), and the decoder is used to gradually restore and refine these features; the head is used to make predictions on specific tasks (such as image segmentation tasks), such as the generation of segmentation masks.

[0042] In one example, each computing node includes at least one GPU (Graphics Processing Unit).

[0043] Accordingly, in this example, the encoder and decoder each include a three-dimensional convolution operator (e.g. Figure 3 The three-dimensional convolution operator (Sliceconv) in .

[0044] The method for generating an image segmentation model provided in an embodiment of the present application is to perform distributed training on an image sample set and a segmentation mask using a master node and multiple computing nodes in a distributed system, wherein the master node distributes each image block to a corresponding computing node, and the image segmentation model is deployed on each computing node by a convolution operator to process the image information of each image block and the boundary information of adjacent image blocks in parallel, thereby completing the training of the image segmentation model, solving the problem of high-performance training of massive image samples, shortening the training time, and improving the training efficiency. Among them, distributed training is a method for training an image segmentation model in parallel on multiple computing nodes. It distributes the model parameters of the image blocks and the image segmentation model to multiple computing nodes, and uses multiple computing nodes to accelerate the training process of the image segmentation model. Among them, the model parameters of the image segmentation model generally refer to the numerical values ​​that define the internal structure and behavior of the model, and these parameters are optimized through the training process to minimize the loss function and improve the numerical values ​​of the model performance, such as weights and biases.

[0045] It should be further explained that the embodiments of the present application are based on a communication mechanism, such as a parameter server model or an All-Reduce algorithm, to synchronously update the model parameters of the image segmentation model.

[0046] The above step S201, namely "obtaining an image sample set, each image sample being marked with a segmentation mask", is described in detail below in conjunction with an embodiment.

[0047] The embodiment of the present application can obtain an image sample set from an external data source (for example, a database, a file system, or a remote server), and the image sample set is a data set for training an image segmentation model. The data set contains the input features (Input Features) of the image segmentation model and the corresponding output labels (i.e., the segmentation mask annotated on the image sample set). The image segmentation model obtains a trained image segmentation model by learning the relationship between the features and labels in the image sample set. Among them, each image sample can be a single image or a frame image in a video, and each image sample can be an image annotated with a segmentation mask on the original image obtained by a certain device in certain fields, wherein the original image is an image such as an X-ray taken by a medical device, a computed tomography (CT) scan, a magnetic resonance imaging (MRI), etc., which is used for medical diagnosis and research. Each image sample is a three-dimensional image, for example, a three-dimensional medical image.

[0048] Here, by classifying and labeling each pixel in the image sample, the fine-grained division of the region in the image sample is achieved. Each pixel is assigned a label to indicate whether it belongs to the foreground, background, or object category in the image sample. The labels of these pixels eventually form a two-dimensional matrix, namely the segmentation mask. The generation of the segmentation mask relies on the technical support of deep learning and artificial intelligence. The image segmentation model is trained through a large number of image samples to achieve accurate segmentation of the image to be segmented.

[0049] In one example, an image annotation tool, such as VIA (VGG Image Annotator), provides users with the ability to manually draw polygons to annotate regions in image samples to generate segmentation masks; or, each image sample is segmented using a deep learning model (such as U-Net, DeepLabv, etc.) to generate a segmentation mask. Among them, U-net is a deep learning architecture commonly used in medical image segmentation, and it is named after its U-shaped network structure. U-net consists of two symmetrical left and right parts, with connections across different depths in the middle. On the left is an encoder, which is used to capture the image features of each image sample, and on the right is a decoder, which is used to map these features back to the size of each image sample.

[0050] In one example, before executing step S202, the method for generating an image segmentation model further includes: preprocessing the image sample set, for example, format conversion, size adjustment, noise removal, etc.

[0051] By preprocessing the image sample set, the noise in each image sample can be removed, the contrast between different tissues and structures in each image sample can be improved, and the edge of the image can be strengthened, thereby improving the image quality of each image sample and providing a more accurate and reliable basis for subsequent image segmentation, feature extraction, diagnosis and analysis.

[0052] In one example, before acquiring the image sample set, the method for generating an image segmentation model further includes:

[0053] Initialize the training environment of the image segmentation model, which includes the number of computing nodes, as well as the address information, identification information and performance data of the master node; establish communication connections between multiple computing nodes and the master node.

[0054] For example, in distributed training, the training environment of the image segmentation model is first initialized, and a communication connection is established between multiple computing nodes and the master node. The training environment includes the number of nodes (world_size), identification information (such as node number (rank)), the address information of the master node (master_addr), performance data, etc.

[0055] The above step S202, namely "dividing each image sample into a plurality of image blocks, and respectively allocating each image block to a corresponding computing node", is described in detail below in conjunction with an embodiment.

[0056] In the embodiment of the present application, each image sample is divided into three-dimensional blocks (for example Figure 4 ) to obtain multiple three-dimensional image blocks. Three-dimensional blocking refers to dividing each image sample (such as a three-dimensional medical image) into multiple three-dimensional image blocks according to certain rules. These image blocks are fixed in size, shape and position, and each image block contains a part of the information in the image. Through three-dimensional blocking, complex images can be divided into small units that are easier to process, thereby simplifying subsequent processing steps. Each image block corresponds to a local area in each image sample.

[0057] The embodiment of the present application can process each image sample individually or process the image sample set in batches to obtain multiple image blocks. Processing the image sample set in batches includes:

[0058] The image sample set is divided into batches to obtain multiple batches of image samples; each image sample in the multiple batches of image samples is divided into a corresponding plurality of image blocks.

[0059] In one example, dividing each image sample into a plurality of image blocks may include: performing edge detection on each image sample, and using the detected boundary information as boundary information between the image blocks.

[0060] The above-mentioned edge detection refers to identifying edges by using changes in pixel intensity in an image. Common edge detection methods include but are not limited to Sobel, Prewitt, and the like.

[0061] In another example, dividing each image sample into a plurality of image blocks may include: using image processing software, such as Adobe image processing software, GIMP (GNU Image Manipulation Program), to divide each image sample into a plurality of image blocks.

[0062] Continuing from the above, three-dimensional blocking refers to dividing each image sample along the three dimensions of depth, height and width to obtain multiple image blocks. Specifically: each image sample is blocked along the three dimensions of depth, height and width to obtain multiple image patches.

[0063] Among them, depth refers to the number of color channels of each image sample. For example, the depth of an RGB image is 3, corresponding to the three channels of R (red), G (green), and B (blue); for example, the depth of an RGBA image is 4, corresponding to the four channels of R (red), G (green), B (blue), and A (transparency channel). Height refers to the number of pixels in the vertical direction of each image sample, which determines the size of each image sample from top to bottom; width refers to the number of pixels in the horizontal direction of each image sample, which determines the size of each image sample from top to bottom.

[0064] The following is a detailed description of “assigning multiple image blocks to corresponding computing nodes”.

[0065] In one example, the master node evaluates the computing resources (such as CPU, GPU, memory, etc.) and communication capabilities of each computing node; then, based on the evaluation results, the master node selects computing nodes with sufficient resources and stable performance to participate in the training of the image segmentation model.

[0066] In another example, multiple image blocks are allocated to corresponding computing nodes according to the number of computing nodes and performance data of the master node.

[0067] The performance data of the master node may include but is not limited to CPU (Central Processing Unit) usage, memory usage, disk I / O (Input / out) speed, and network bandwidth. These performance data are crucial for evaluating the load capacity of the master node and formulating load balancing strategies. In distributed training, load balancing can ensure the overall performance and reliability of the distributed system.

[0068] Continuing from the above, the computing node includes at least one GPU, and each image block is respectively allocated to a corresponding GPU in the corresponding computing node.

[0069] Continuing from the above, when processing each image sample individually, each GPU processes an image block independently; when processing image sample sets in batches, there is a dependency between image samples in multiple batches, and it is necessary to exchange boundary information across GPUs.

[0070] The computing nodes in the embodiments of the present application are usually equipped with high-performance processors, large-capacity content and high-speed storage devices to improve their computing speed and processing capabilities. Therefore, each image block is assigned to a corresponding computing node, and each computing node trains the image segmentation model in parallel, which can shorten the training time and improve the training efficiency.

[0071] The following is a detailed description of step S203, i.e., "using the image sample set and the segmentation mask to train the image segmentation model on the computing node; wherein the feature extraction processing involved in the training process includes: processing the image information and boundary information of the assigned image blocks in parallel through the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes assigned adjacent image blocks" in conjunction with the embodiments.

[0072] The embodiment of the present application uses multiple computing nodes to train the image segmentation model in parallel using the image sample set and the segmentation mask. The feature extraction process involved in the training of the image segmentation model includes: the image information and boundary information of the assigned image block are processed in parallel by the convolution operator on each computing node to obtain the local features corresponding to each image block, and the boundary information comes from the computing node assigned the adjacent image block.

[0073] In the process of training image segmentation models, feature extraction is used to extract features of each image block, thereby achieving accurate image segmentation. In order to obtain higher resolution segmentation results, upsampling and downsampling operations are usually used. Among them, downsampling reduces the resolution of the image through pooling or step operations to better capture features. Upsampling restores the features to the original resolution through methods such as deconvolution or interpolation.

[0074] The image information of an image block refers to the pixel values ​​and their distribution characteristics in a local area of ​​each image sample. In the convolution processing by the convolution operator, the image block is usually obtained by moving a sliding window (also called a convolution window) on each image sample. Each time it moves, the pixel values ​​in the window constitute the image block currently being processed.

[0075] Boundary information refers to the boundary or edge information between different image blocks in each image sample. In image processing, boundary information is an important basis for distinguishing different image blocks. In convolution processing through convolution operators, since the size of the convolution kernel is usually fixed (such as 3x3, 5x5, etc.), when processing the boundary of the image block, the boundary pixels may not completely correspond to the convolution kernel. In order to solve this problem, it is usually necessary to process the boundary, such as zero padding, mirror padding or copy padding, to ensure the quality and accuracy of the image output by the convolution processing.

[0076] Continuing from the above, before each computing node sends boundary information to its adjacent computing nodes, each computing node can extract the boundary information features to obtain boundary features; next, each computing node sends the boundary features to its adjacent computing nodes for convolution processing; wherein, the boundary features usually indicate the end of one local area and the beginning of another local area, and the features or attributes in the local areas separated by these boundaries are consistent, while the features or attributes in different local areas are different. Before the convolution processing, through the boundary features between adjacent image blocks, the image segmentation model can better capture the structural information in the image and provide strong support for subsequent image segmentation tasks.

[0077] The embodiment of the present application performs convolution processing on the image information and boundary information of each image block through a convolution operator, so as to obtain a local feature map containing boundary information.

[0078] In the embodiment of the present application, before performing convolution processing by the convolution operator, the boundary information of adjacent image blocks is obtained through communication between the computing nodes (for example, Figure 4 The boundary information is shown in FIG. 1 ), and the boundary information is added to the edge of the allocated image block, thereby completing the boundary filling of the allocated image block.

[0079] As mentioned above, each computing node includes at least one GPU. Figure 4 Before the convolution shown in FIG. 1 , the method for generating an image segmentation model further includes:

[0080] Through communication between GPUs (for example, Figure 4 For example, the GPU assigned an image block communicates with the GPU assigned an adjacent image block to obtain boundary information from the GPU assigned an adjacent image block, and processes the image information and boundary information of the assigned image block in parallel through the convolution operator on the GPU assigned the image block to obtain the local features corresponding to the assigned image block. Next, the image segmentation model is trained in parallel on multiple GPUs using the local features and the segmentation mask.

[0081] In one example, the feature extraction process specifically includes: determining the global features corresponding to each image sample based on the local features of each image block; and training the image segmentation model on multiple computing nodes using the global features and segmentation masks corresponding to each image sample.

[0082] Here, the following methods can be used, such as splicing, weighted averaging, principal component analysis (PCA) dimensionality reduction, etc. to fuse the local features of each image block to obtain a global feature, which can describe the overall layout, color distribution, texture and other characteristics of each image sample.

[0083] In the embodiment of the present application, when communicating between the computing nodes, it is necessary to ensure that when the boundary of the image block assigned by the convolution processing is performed, the boundary information from the adjacent image block can be received in time, while avoiding communication deadlock and conflict.

[0084] Continuing from the above, before using global features to train the image segmentation model, the global parameters of the image segmentation model are synchronously updated through a communication mechanism, such as a parameter server model or an All-Reduce algorithm; wherein the global parameters generally refer to the parameters in the image segmentation model that affect the behavior or internal structure of the entire model.

[0085] In one example, before obtaining the local features of each image block, the method for generating an image segmentation model further includes:

[0086] Performing parity sorting on multiple computing nodes, and determining communication strategy information between the computing nodes according to the sorting result, the communication strategy information is used to characterize the communication time and communication mode corresponding to each computing node in the sorting result, the communication time is the time when each computing node transmits boundary information using the communication mode, and the communication mode is receiving or sending;

[0087] The communication between the computing nodes is controlled based on the communication strategy information, so that each computing node can respectively obtain the boundary information of the allocated image block.

[0088] The embodiment of the present application determines the communication time and communication mode between the computing nodes through the sorting result of the odd-even sorting.

[0089] For example, multiple computing nodes are sorted by odd and even numbers, and the computing nodes on odd numbers are controlled to send boundary information to adjacent computing nodes in parallel; next, the computing nodes on even numbers are controlled to send boundary information to adjacent computing nodes in parallel; until multiple computing nodes complete the exchange of boundary information.

[0090] It should be noted that after a computing node on an odd-numbered position sends boundary information to its adjacent computing node, when it is the adjacent computing node's turn to send boundary information, it must ensure that the adjacent computing node has received the boundary sent by the computing node on the odd-numbered position to avoid communication conflicts.

[0091] Among them, Odd-Even Sort is a method suitable for distributed systems. It allows adjacent computing nodes to exchange boundary information each time through multiple iterations. In a distributed system, multiple computing nodes are controlled to receive or send boundary information to their adjacent computing nodes in parallel according to the sorting results, which can reduce communication overhead.

[0092] In the embodiment of the present application, during the convolution operation of the convolution operator, the above-mentioned communication strategy is adopted to exchange boundary information between image blocks, so that rich global context information can be retained in the features after convolution processing, thereby enhancing the overall performance of the image segmentation model and improving the accuracy of image segmentation.

[0093] Continuing from the above, each computing node includes at least one GPU, and multiple GPUs in each computing node are sorted by odd-even sorting, and communication strategy information between each GPU is determined according to the sorting result, and the communication strategy information is used to characterize the communication time and communication mode corresponding to each GPU, and the communication time is the time when each GPU adopts the communication mode to transmit boundary information, and the communication mode is receiving or sending; next, the communication time and communication mode corresponding to each GPU in the sorting result are characterized by the communication strategy information, and the communication time is the time when each computing node adopts the communication mode to transmit boundary information, and the communication mode is receiving or sending, so that the boundary information can be exchanged between adjacent GPUs (that is, the two GPUs where two adjacent image blocks are located).

[0094] The embodiment of the present application adopts odd-even sorting of multiple GPUs to determine the communication time and communication method of multiple GPUs exchanging boundaries with their adjacent GPUs in parallel, which can reduce communication overhead.

[0095] Figure 5 A flowchart of a method for generating an image segmentation model provided in an embodiment of the present application, wherein the scheme is applied to Figure 1 The following is an example of the master node in Figure 1. Figure 5 As shown in , the method may include the following steps:

[0096] Step S501: Obtain an image to be segmented.

[0097] Step S502: Divide the image to be segmented into a plurality of image blocks, and assign each image block to a corresponding computing node.

[0098] Step S503: Use the image segmentation model to perform segmentation processing on multiple image blocks to obtain segmentation results for the image to be segmented; wherein the image segmentation model includes convolution operators deployed on each computing node respectively, and the feature extraction processing involved in the segmentation processing includes: using the convolution operators on each computing node to parallelly process the image information and boundary information of the assigned image blocks to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes assigned adjacent image blocks.

[0099] It can be seen that the embodiment of the present application uses the convolution operator deployed on each computing node of the image segmentation model to parallelly process the image information and boundary information of the assigned image blocks, obtain the local features of each pair of image blocks in the image to be segmented, and thus accurately complete the segmentation of the image to be segmented.

[0100] The following describes in detail each step in the above process and the effects that can be further produced in conjunction with the embodiments.

[0101] First, the above step S501, namely "obtaining the image to be segmented", is described in detail in conjunction with the embodiment.

[0102] The image to be segmented in the embodiment of the present application is the image to be segmented. The image to be segmented can be an original image obtained by a certain device in certain fields, such as X-rays, computed tomography (CT) scans, magnetic resonance imaging (MRI) and other images taken by medical equipment for medical diagnosis and research. The image to be segmented can also be an image obtained after the original image is processed by certain image processing techniques, such as an image obtained after cropping, rotation and other processing. The image to be segmented can be in a variety of image formats, such as JPEG / JPG, PNG, etc. The image to be segmented can be a three-dimensional image, for example, a three-dimensional medical image.

[0103] The above step S502, namely "dividing the image to be segmented into a plurality of image blocks, and respectively allocating each image block to a corresponding computing node", is described in detail below in conjunction with an embodiment.

[0104] The embodiment of the present application performs three-dimensional segmentation on the image to be segmented to obtain multiple three-dimensional image blocks. Among them, three-dimensional segmentation refers to dividing the image to be segmented (such as a three-dimensional medical image) into multiple three-dimensional image blocks according to certain rules. These image blocks are fixed in size, shape and position, and each image block contains a part of the information in the image. Through three-dimensional segmentation, complex images can be segmented into small units that are easier to process, thereby simplifying subsequent processing steps.

[0105] Continuing from the above, three-dimensional blocking refers to dividing the image to be segmented along the three dimensions of depth, height and width to obtain multiple image blocks. Specifically: the image to be segmented is blocked along the three dimensions of depth, height and width to obtain multiple image patches.

[0106] It should be noted that the step of dividing the image into a plurality of image blocks may refer to the description in the above step S202 and will not be repeated here.

[0107] The following describes in detail the above step S503, i.e., "using an image segmentation model to perform segmentation processing on multiple image blocks to obtain segmentation results for the image to be segmented; wherein the image segmentation model includes convolution operators respectively deployed on each computing node, and the feature extraction processing involved in the segmentation processing includes: using the convolution operators on each computing node to parallelly process the image information and boundary information of the assigned image blocks to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes assigned adjacent image blocks" in conjunction with the embodiments.

[0108] The embodiment of the present application performs an image segmentation operation on the image to be segmented. Image segmentation is an important task in the field of computer vision. Its goal is to separate and identify different objects or regions in an image based on the characteristics of the image, and to divide the digital image into multiple regions or pixel sets with semantic information.

[0109] The embodiment of the present application obtains the segmentation result according to the output of the image segmentation model. According to different image segmentation model structures, the output result of the image segmentation model may be in various data formats, such as pixel-level label map, probability map and bounding box.

[0110] As a preferred implementation, when the image segmentation model includes an encoder, a decoder, and a head, and the encoder and the decoder respectively include convolution operators deployed on each computing node, the output is a segmentation mask with the same size as the input image. Each pixel in the segmentation mask corresponds to a category label, indicating the object category to which the pixel belongs.

[0111] It should be noted that the image segmentation model is obtained by training in the above-mentioned method (i.e., the method for generating an image segmentation model) embodiment, and the contents related to the image segmentation model are the same as those described in the above-mentioned method embodiment and will not be repeated here.

[0112] Combination of the above Figure 2 , describes the method embodiment of the present application in detail, and the following is combined with Figure 6 , describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0113] Figure 6 FIG. 1 is a schematic diagram of the structure of a device for generating an image segmentation model provided by an embodiment of the present application. Figure 6As shown, the device for generating an image segmentation model provided in an embodiment of the present application includes a sample acquisition module 601, an image processing module 602 and a model training module 603. The image segmentation model includes convolution operators respectively deployed on each computing node, and the device includes: a sample acquisition module 601, which is used to acquire an image sample set, each image sample is marked with a segmentation mask; an image processing module 602, which is used to divide each image sample into multiple image blocks, and respectively assign each image block to a corresponding computing node; a model training module 603, which is used to train the image segmentation model on the computing node using the image sample set and the segmentation mask; wherein the feature extraction processing involved in the training process includes: the image information and boundary information of the assigned image block are processed in parallel by the convolution operator on each computing node to obtain the local features corresponding to each image block, and the boundary information comes from the computing node assigned the adjacent image block.

[0114] In one embodiment, the feature extraction process includes, specifically, determining a global feature corresponding to each image sample based on local features of each image block; and training an image segmentation model on a computing node using the global feature corresponding to each image sample and a segmentation mask.

[0115] In one embodiment, the device also includes: a communication strategy determination module (not shown in the figure), which is used to sort the computing nodes by odd and even numbers, and determine the communication strategy information between each computing node based on the sorting result, the communication strategy information is used to characterize the communication time and communication mode corresponding to each computing node in the sorting result, the communication time is the time when each computing node adopts the communication mode to transmit boundary information, and the communication mode is receiving or sending; an information acquisition module (not shown in the figure), which is used to control the communication between each computing node through the communication strategy information, so that each computing node can obtain the boundary information of the assigned image block respectively.

[0116] In one embodiment, the device is applied to a master node, and the device further includes:

[0117] An initialization module is used to initialize the training environment of the image segmentation model. The training environment includes the number of computing nodes, as well as the address information, identification information and performance data of the master node;

[0118] A communication module is established to establish a communication connection between the computing node and the master node.

[0119] In one embodiment, the image processing module 602 is specifically used to: allocate multiple image blocks to corresponding computing nodes respectively according to the number of computing nodes and the performance data of the master node.

[0120] In one embodiment, the structure of the image segmentation model includes an encoder, a decoder and a head, and the encoder and the decoder respectively include convolution operators deployed on each computing node.

[0121] Combination of the above Figure 5 , describes in detail the device embodiment of the present application, and the following is combined with Figure 7 , describes the device embodiment of the present application in detail. It should be understood that the description of the device embodiment corresponds to the description of the device embodiment, so the part that is not described in detail can refer to the previous device embodiment.

[0122] Figure 7 FIG. 1 is a schematic diagram of the structure of a device for generating an image segmentation model provided by an embodiment of the present application. Figure 7 As shown, the apparatus for generating an image segmentation model provided in the embodiment of the present application includes an image acquisition module 701, an image processing module 702 and an image segmentation module 703. The image acquisition module 701 is used to acquire the image to be segmented; the image processing module 702 is used to divide the image to be segmented into multiple image blocks, and respectively assign each image block to a corresponding computing node; the image segmentation module 703 is used to perform segmentation processing on multiple image blocks using the image segmentation model to obtain a segmentation result for the image to be segmented; the image segmentation model includes convolution operators respectively deployed on each computing node, and the feature extraction processing involved in the segmentation processing includes: the image information and boundary information of the assigned image blocks are processed in parallel by the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes assigned adjacent image blocks.

[0123] Below, reference Figure 8 To describe an electronic device according to an embodiment of the present application. Figure 8 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application.

[0124] like Figure 8 As shown, the electronic device includes one or more processors 801 and a memory 802 .

[0125] The processor 801 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0126] The memory 802 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 801 may run the program instructions to implement the functions of the method for generating an image segmentation model or the method for image segmentation of the various embodiments of the present application mentioned above. Image samples, etc. may also be stored in the computer-readable storage medium.

[0127] In one example, the electronic device may further include: an input device 803 and an output device 804, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0128] The input device 803 may include, for example, a keyboard, a mouse, etc.

[0129] The output device 804 can output various information to the outside, including segmentation results, etc. The output device 804 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0130] Of course, to simplify, Figure 8 Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.

[0131] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the method for generating an image segmentation model or the method for image segmentation according to various embodiments of the present application described above in this specification.

[0132] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and also conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user device, as an independent software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server.

[0133] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps in the method for generating an image segmentation model or the method for image segmentation according to various embodiments of the present application described above in this specification.

[0134] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0135] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0136] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0137] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0138] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0139] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0140] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating an image segmentation model, wherein: The image segmentation model includes convolution operators respectively deployed on each computing node, and the method includes: Obtain an image sample set, each image sample is marked with a segmentation mask; Divide each image sample into multiple image blocks, and assign each image block to a corresponding computing node; The image segmentation model is trained on the computing node using each image sample and the segmentation mask; wherein the feature extraction processing involved in the training process includes: processing the image information and boundary information of the assigned image blocks in parallel through the convolution operator on each computing node to obtain the local features corresponding to each image block, wherein the boundary information comes from the computing node assigned the adjacent image blocks.

2. The method according to claim 1, wherein: The feature extraction process includes: Determine the global features corresponding to each image sample according to the local features of each image block; An image segmentation model is trained on the computing node using the global features corresponding to each image sample and the segmentation mask.

3. The method according to claim 1 or 2, wherein: Before obtaining the local features of each image block, the method further includes: Performing parity sorting on the computing nodes, and determining communication strategy information between the computing nodes according to the sorting result, wherein the communication strategy information is used to characterize the communication time and communication mode corresponding to each computing node in the sorting result, wherein the communication time is the time when each computing node transmits boundary information using the communication mode, and the communication mode is receiving or sending; The communication strategy information is used to control the communication between the computing nodes, so that each computing node can obtain the boundary information of the allocated image block.

4. The method according to claim 1, wherein: The method is applied to the master node, and before acquiring the image sample set, the method further includes: Initializing a training environment of the image segmentation model, the training environment including the number of computing nodes, and address information, identification information, and performance data of the master node; A communication connection is established between the computing node and the master node.

5. The method according to claim 4, wherein the step of respectively allocating each image block to a corresponding computing node comprises: According to the number of the computing nodes and the performance data of the master node, multiple image blocks are respectively allocated to corresponding computing nodes.

6. The method according to claim 1, wherein: The structure of the image segmentation model includes an encoder, a decoder and a head, and the encoder and the decoder respectively include convolution operators deployed on each computing node.

7. A method for image segmentation, characterized in that: The method comprises: Obtain the image to be segmented; Dividing the image to be segmented into a plurality of image blocks, and assigning each image block to a corresponding computing node; The multiple image blocks are segmented using an image segmentation model to obtain a segmentation result for the image to be segmented; wherein the image segmentation model includes convolution operators deployed on each computing node respectively, and the feature extraction processing involved in the segmentation processing includes: parallel processing of image information and boundary information of the assigned image blocks by the convolution operators on each computing node to obtain local features corresponding to each image block, wherein the boundary information comes from the computing nodes to which adjacent image blocks are assigned.

8. A device for generating an image segmentation model, wherein: The image segmentation model includes convolution operators respectively deployed on each computing node, and the device includes: A sample acquisition module is used to acquire an image sample set, each image sample is marked with a segmentation mask; An image processing module, used for dividing each image sample into a plurality of image blocks, and respectively allocating each image block to a corresponding computing node; A model training module is used to train an image segmentation model on the computing node using the image sample set and the segmentation mask; wherein the feature extraction processing involved in the training process includes: processing the image information and boundary information of the assigned image blocks in parallel through the convolution operator on each computing node to obtain local features corresponding to each image block, wherein the boundary information comes from the computing node assigned the adjacent image block.

9. An image segmentation device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be segmented; An image processing module, used for dividing the image to be segmented into a plurality of image blocks, and respectively allocating each image block to a corresponding computing node; An image segmentation module is used to perform segmentation processing on the multiple image blocks using an image segmentation model to obtain a segmentation result for the image to be segmented; wherein the image segmentation model includes convolution operators deployed on each computing node respectively, and the feature extraction processing involved in the segmentation processing includes: processing the image information and boundary information of the assigned image blocks in parallel through the convolution operators on each computing node to obtain local features corresponding to each image block, and the boundary information comes from the computing nodes assigned adjacent image blocks.

10. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 8 by running the program in the memory.

11. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images