Ultrasound image segmentation model training method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide an ultrasound image segmentation model training method and device, electronic equipment and storage medium, applied to the field of medical imaging technology. By taking N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes in the image segmentation network, and disconnecting or reconnecting the connection relationship between each node with a probability P, an image segmentation network is generated; the image segmentation network is connected with the tolerance generation network to obtain an ultrasound image segmentation model and is trained. By applying the method of the embodiments of the present application, the connection between each node can be disconnected or reconnected by the probability P, the image segmentation network is generated, the image segmentation network is connected with the tolerance generation network, the ultrasound image segmentation model is generated, so that the ultrasound image segmentation model can be trained and applied to ultrasound image segmentation, and then the automatic segmentation detection of ultrasound images is realized, without manual segmentation, reducing the cost of ultrasound image segmentation.

Description

Technical Field

[0001] This application relates to the field of medical imaging technology, and in particular to methods, devices, electronic equipment and storage media for training ultrasound image segmentation models. Background Technology

[0002] Ultrasound imaging is an examination method that uses the echo information generated by the propagation of high-frequency ultrasound in human tissues to display superficial soft tissue structures such as muscles, tendons, ligaments, and peripheral nerves. It can be applied to the diagnosis, monitoring, and efficacy evaluation of various joint and muscle diseases, and has advantages such as real-time, non-invasiveness, simple operation, and no radiation.

[0003] However, ultrasound examinations require doctors with certain expertise to differentiate between different tissue areas in order to observe different tissues. The training period for doctors is relatively long. Therefore, how to achieve automatic detection of ultrasound images has become a technical problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for training an ultrasound image segmentation model, so as to achieve automatic detection of ultrasound images. The specific technical solution is as follows:

[0005] In a first aspect of this application, a method for training an ultrasound image segmentation model is provided, the method comprising:

[0006] Obtain the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1;

[0007] N is the number of nodes in the image segmentation network, K is the number of connections between each node and other nodes in the image segmentation network, and the connection between each node is disconnected or reconnected with probability P to generate the image segmentation network.

[0008] Obtain the tolerance generation network, and connect the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained;

[0009] The ultrasound image segmentation model is trained using sample ultrasound images.

[0010] In one possible implementation, the step of generating the image segmentation network by using N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes in the image segmentation network, and disconnecting or reconnecting the connections between nodes with probability P, includes:

[0011] Generate N nodes, where the i-th node includes a convolutional layer, the (i+1)-th node includes a convolutional layer and a pooling layer, and i is an even number between 0 and N-1;

[0012] For each node, establish connections between that node and the other K nodes;

[0013] The image segmentation network is obtained by determining whether to disconnect nodes with a probability of P and whether to connect nodes without a probability of P, and by performing the determination results.

[0014] In one possible implementation, establishing connections between each node and the other K nodes includes:

[0015] For each node, a connection is established between that node and its K nearest neighbors.

[0016] In one possible implementation, the method further includes:

[0017] When a first type of node exists, a connection is established between the first type of node and at least one node ordered before it, wherein the first type of node is a node that has not established a connection with any node ordered before it.

[0018] In one possible implementation, the size of the i-th node is:

[0019]

[0020] In one possible implementation, connecting the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained includes:

[0021] The output of the (N-1) / 2th node of the image segmentation network is connected to the input of the tolerance generation network, and the output of the tolerance generation network is connected to the output of the image segmentation network to obtain the ultrasound image segmentation model to be trained.

[0022] In one possible implementation, training the ultrasound image segmentation model using sample ultrasound images includes:

[0023] The sample ultrasound image is input into the ultrasound image segmentation model to obtain the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network; wherein, the sample ultrasound image corresponds to a segmentation label, and the segmentation label represents the true value of the sample ultrasound image segmentation result.

[0024] The predicted segmentation result is obtained by multiplying the elements at the same position in the predicted probability map and the predicted tolerance map.

[0025] Calculate the segmentation label of the sample ultrasound image and the loss of the predicted segmentation result;

[0026] The parameters of the ultrasound image segmentation model are adjusted based on the loss.

[0027] In one possible implementation, the method further includes:

[0028] Based on the loss, adjust the connection weights between connected nodes in the image segmentation network;

[0029] For any given connection, disconnect the connection if the connection weight is less than a preset weight threshold.

[0030] In a second aspect of this application, an ultrasound imaging detection method is provided, the method comprising:

[0031] Acquire the image to be detected;

[0032] The image to be detected is input into an image segmentation network to obtain a segmentation result, wherein the image segmentation network is trained by any of the methods described in the first aspect of the embodiments of this application.

[0033] In a third aspect of this application, an ultrasound image segmentation model training device is provided, the device comprising:

[0034] The parameter acquisition module is used to obtain the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1.

[0035] The node connection module is used to take N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes in the image segmentation network, and disconnect or reconnect the connections between nodes with a probability of P to generate the image segmentation network.

[0036] The network connection module is used to obtain the tolerance generation network and connect the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained.

[0037] The model training module is used to train the ultrasound image segmentation model using sample ultrasound images.

[0038] In one possible implementation, the node connection module includes:

[0039] The node generation submodule is specifically used to generate N nodes, where the i-th node includes a convolutional layer, the (i+1)-th node includes a convolutional layer and a pooling layer, and i is an even number belonging to 0 to N-1;

[0040] The node connection submodule is specifically used to establish connections between each node and the other K nodes.

[0041] The connection determination submodule is specifically used to determine whether to disconnect nodes with a probability of P, and to determine whether to connect nodes without a probability of P, and to obtain the image segmentation network by executing the determination results.

[0042] In one possible implementation, the node connecting sub-modules includes:

[0043] The node connection unit is specifically used to establish connections between each node and its K nearest neighbors.

[0044] In one possible implementation, the device further includes:

[0045] The first type of node connection module is used to establish a connection between the first type of node and at least one node that is ordered before it when a first type of node exists, wherein the first type of node is a node that has not established a connection with any node that is ordered before it.

[0046] In one possible implementation, the size of the i-th node is:

[0047]

[0048] In one possible implementation, the network connectivity module includes:

[0049] The network connection submodule is specifically used to connect the output of the (N-1) / 2th node of the image segmentation network with the input of the tolerance generation network, and to connect the output of the tolerance generation network with the output of the image segmentation network to obtain the ultrasound image segmentation model to be trained.

[0050] In one possible implementation, the model training module includes:

[0051] The sample ultrasound image input submodule is specifically used to input the sample ultrasound image into the ultrasound image segmentation model to obtain the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network; wherein, the sample ultrasound image corresponds to a segmentation label, and the segmentation label represents the ground truth value of the sample ultrasound image segmentation result;

[0052] The product operation submodule is specifically used to perform a product operation on elements at the same position in the prediction probability map and the prediction tolerance map to obtain the prediction segmentation result;

[0053] The loss calculation submodule is specifically used to calculate the segmentation label of the sample ultrasound image and the loss of the predicted segmentation result;

[0054] The parameter adjustment submodule is specifically used to adjust the parameters of the ultrasound image segmentation model based on the loss.

[0055] In one possible implementation, the device further includes:

[0056] A weight adjustment module is used to adjust the connection weights between connected nodes in the image segmentation network according to the loss.

[0057] The connection disconnection module is used to disconnect any connection if the connection weight of that connection is less than a preset weight threshold.

[0058] In a fourth aspect of this application, an ultrasound imaging detection device is provided, the device comprising:

[0059] The image acquisition module is used to acquire images to be detected.

[0060] An ultrasound image detection module is used to input the image to be detected into an image segmentation network to obtain a segmentation result, wherein the image segmentation network is trained by any of the methods described in the first aspect of the embodiments of this application.

[0061] In a fifth aspect of the embodiments of this application, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0062] Memory, used to store computer programs;

[0063] When a processor executes a program stored in memory, it implements the steps of the method described in either the first or second aspect of the embodiments of this application.

[0064] In a sixth aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the method steps of either the first or second aspect of the present application.

[0065] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the ultrasound image segmentation model training methods and ultrasound image detection methods described above.

[0066] Beneficial effects of the embodiments in this application:

[0067] This application provides a method, apparatus, electronic device, and storage medium for training an ultrasound image segmentation model. The method involves obtaining the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1. N is used as the number of nodes in the image segmentation network, K is used as the number of connections between each node and other nodes in the image segmentation network, and connections between nodes are broken or reconnected with probability P to generate the image segmentation network. A tolerance generation network is then obtained, and the image segmentation network is connected to the tolerance generation network to obtain the ultrasound image segmentation model to be trained. The ultrasound image segmentation model is trained using sample ultrasound images. By applying the method of this application, an image segmentation network can be generated by breaking or reconnecting connections between nodes with probability P. Connecting the image segmentation network to the tolerance generation network generates an ultrasound image segmentation model. This allows for training the ultrasound image segmentation model and using the trained model for ultrasound image segmentation, thereby achieving automated segmentation and detection of ultrasound images without manual segmentation, reducing the cost of ultrasound image segmentation.

[0068] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0070] Figure 1 A flowchart of an ultrasound image segmentation model training method provided in an embodiment of this application;

[0071] Figure 2 This is a schematic diagram of the connection of image segmentation network nodes provided in an embodiment of this application;

[0072] Figure 3 This is another schematic diagram illustrating the connection of image segmentation network nodes provided in an embodiment of this application;

[0073] Figure 4 A schematic diagram illustrating the inter-node image size conversion provided in an embodiment of this application;

[0074] Figure 5 A schematic diagram illustrating the connection of an ultrasound image segmentation model provided in an embodiment of this application;

[0075] Figure 6 A schematic diagram of generating a prediction tolerance map provided in an embodiment of this application;

[0076] Figure 7 A schematic diagram illustrating the disconnection of a connection with a smaller weight provided in an embodiment of this application;

[0077] Figure 8 A flowchart of an ultrasound imaging detection method provided in an embodiment of this application;

[0078] Figure 9 A schematic diagram of a structure of an ultrasound image segmentation model training device provided in an embodiment of this application;

[0079] Figure 10 This is a schematic diagram of the structure of an ultrasonic imaging detection device provided in an embodiment of this application;

[0080] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0082] Musculoskeletal ultrasound is an emerging ultrasound examination technology in recent years. By applying high-frequency ultrasound, it can clearly display superficial soft tissue structures such as muscles, tendons, ligaments, and peripheral nerves, thereby enabling the diagnosis, monitoring, and efficacy evaluation of various joint and muscle diseases. It offers advantages such as real-time monitoring, non-invasiveness, ease of operation, and no radiation exposure. However, the training period for physicians is long, and diagnostic standards are inconsistent, leading to significant subjectivity and inconsistent ultrasound image segmentation results, as well as high costs. Furthermore, segmented ultrasound images, similar to other two-dimensional ultrasound images, suffer from high noise, blurred regions, weak boundaries, and difficulty in locating regions of interest (ROIs). This results in low accuracy for automated ultrasound image segmentation techniques, while manual segmentation of the target region is extremely labor-intensive. Therefore, utilizing musculoskeletal ultrasound segmentation technology to achieve automated target segmentation is of great significance.

[0083] In related technologies, ultrasound image segmentation models are based on an improved encoder-decoder network architecture of Unet (U-net, U-shaped deep learning architecture), often using regular convolutions, requiring manual design of network connections. Another possible implementation is to attempt to automatically design the network architecture through Neural Architecture Search (NAS), but this method requires a large amount of computing hardware resources and is inefficient.

[0084] Furthermore, when interpreting musculoskeletal ultrasound images, it is necessary to segment different tissue regions. Semantic segmentation algorithms rely on annotations from professional technicians. However, for segmentation tasks with ambiguous boundaries, different experts may use different annotations, causing uncertain annotation boundaries to interfere with the overall performance of the ultrasound image segmentation model. In addition, ultrasound examinations involve dynamic images; the continuous changes in the images not only reduce the computational efficiency of the ultrasound image segmentation model but also affect its accuracy.

[0085] To address at least one of the aforementioned technical problems, a first aspect of this application provides a method for training an ultrasound image segmentation model, the method comprising: Figure 1 The steps shown are as follows:

[0086] Step S101: Obtain the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1.

[0087] The ultrasound image segmentation model contains multiple nodes, which are interconnected to form the image segmentation network. To ensure model symmetry and bottom feature output, the number of nodes N is set to an odd number greater than 9, and the number of connections K between a node and its neighboring nodes is set to an integer greater than 1. The number of nodes N, the number of connections K, and the connection probability P can all be customized according to actual conditions. In one example, N is set to 11, 13, 15, 17, etc., K is set to 2, 3, 4, 5, etc., and P is set to 0.1, 0.2, 0.3, 0.4, etc.

[0088] Step S102: Use N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes in the image segmentation network, and disconnect or reconnect the connections between nodes with probability P to generate the image segmentation network.

[0089] Using the example above, when N is 13 and K is 4, a rule network is generated as follows: Figure 2The diagram on the left shows a regular network with 13 nodes, each connected to 4 other nodes. Next, connections between nodes are broken or reconnected with probability P. In one example, if P is 0.4, then for two connected nodes, there is a 0.4 probability of breaking the connection; for two unconnected nodes, there is a 0.4 probability of connecting them. Figure 2 The diagram on the right shows a small-world network, which is an image segmentation network that has been randomly disconnected and reconnected by a network with a probability rule of P.

[0090] Step S103: Obtain the tolerance generation network, and connect the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained.

[0091] The ultrasound image segmentation model consists of a tolerance generation network and an image segmentation network. The image features output by the image segmentation network serve as the input to the tolerance generation network. After processing by the tolerance generation network, a tolerance region map is obtained. Therefore, when connecting the image segmentation network and the tolerance generation network, the output of the image segmentation network can be concatenated with the tolerance generation network to obtain the ultrasound image segmentation model.

[0092] Step S104: Train the ultrasound image segmentation model using sample ultrasound images.

[0093] The sample ultrasound images are ultrasound images with pre-labeled segmentation tags. The segmentation tags represent the true values ​​of the segmentation results of the sample ultrasound images and can be obtained by professionals.

[0094] By applying the method of this application embodiment, an image segmentation network can be generated by disconnecting or reconnecting the connections between nodes with a probability of P. The image segmentation network is then connected to a tolerance generation network to generate an ultrasound image segmentation model. The ultrasound image segmentation model can be trained and used for ultrasound image segmentation, thereby achieving automated detection of ultrasound images without manual segmentation and reducing the cost of ultrasound image segmentation.

[0095] In one possible implementation, step S102 can be accomplished through the following steps:

[0096] Step 1: Generate N nodes, where the i-th node includes a convolutional layer, the (i+1)-th node includes both a convolutional layer and a pooling layer, and i is an even number between 0 and N-1.

[0097] For ease of description, each node can be numbered. In one example, let's take N=13 as an example, such as... Figure 3As shown, there are 13 nodes, from node 0 to node 12, where i is an even number belonging to the range 0 to 12, i.e., 0, 2, 4, 6, 8, 10, 12. For different nodes, convolution operations are assigned: even-numbered nodes only include convolutional layers and perform convolution operations; odd-numbered nodes include both convolutional and pooling layers; and among odd-numbered nodes, nodes with numbers less than N / 2 perform both convolution and pooling operations, while nodes with numbers greater than N / 2 perform both convolution and unpooling operations. That is, in... Figure 3 In the diagram, black represents nodes with odd numbers, and white represents nodes with even numbers. Even nodes 0, 2, 4, 6, 8, 10, and 12 can only include convolutional layers to perform convolution operations; odd nodes 1, 3, and 5 can include convolutional layers and pooling layers to perform convolution and pooling operations; odd nodes 7, 9, and 11 can include convolutional layers and pooling layers (specifically, unpooling layers) to perform convolution and unpooling operations respectively.

[0098] Step 2: For each node, establish connections between that node and the other K nodes.

[0099] For any given node, connections are established between that node and the other K nodes. These other K nodes can be randomly selected, or they can be the K nodes adjacent to the given node, etc., all within the scope of this application. Here, a connection represents the transmission of image features, and the image features are transmitted from the node with the higher ranking to the node with the lower ranking; for example, if node 3 (ranked 3) establishes a connection with node 5 (ranked 5), then the influence features output by node 3 will be used as the input to node 5.

[0100] In one example, taking K=4 as an example, such as Figure 2 As shown in the diagram of the rule network on the left, each node has four other nodes connected to it. A connection represents the transmission of image features, and image features are transmitted from the node ranked higher to the node ranked lower. For example... Figure 3 As shown, nodes with smaller numbers point to nodes with larger numbers, with node 0 being the input and node 12 being the output. In one example, as... Figure 3 As shown in the dashed circle, node 3 represents the input nodes, while nodes 0, 1, and 2 represent the input nodes, and nodes 6 and 9 represent the output nodes.

[0101] In one possible implementation, for each node, connections can be established between that node and its K nearest neighbors. The K nearest neighbors can be determined by numbering, selecting the K nearest neighbors whose numbers are closest to the node's number. Alternatively, the K nearest neighbors can be determined by connecting each node to the node with lines, and the length of these lines determining the K nearest neighbors.

[0102] Step 3: Determine whether to disconnect nodes with a probability of P, and determine whether to connect nodes without a probability of P. The result of the determination is used to obtain the image segmentation network.

[0103] By applying the method of this application embodiment, nodes of a regular network can be randomly disconnected or connected based on probability P, thereby generating a small-world network in this application embodiment. This enables the generation of new networks guided by the small-world theory of complex networks, thereby reducing the complexity of ultrasound image segmentation model design, improving the flexibility of convolutional combination, enhancing the feature representation ability of ultrasound image segmentation models, and ultimately improving the accuracy of ultrasound image segmentation models.

[0104] In practical applications, after disconnecting or reconnecting the connections between nodes with probability P, there will inevitably be nodes that only have outputs and no inputs, as well as nodes that only have inputs and no outputs. Therefore, embodiments of this application may further include the following methods:

[0105] When a first-class node exists, establish a connection between the first-class node and at least one node ordered before it.

[0106] Among them, the first type of node is a node that has not established a connection with any node ranked before it. Therefore, this first type of node is a node with no input, such as... Figure 3 Nodes 1 and 8 are shown. For this type of node, a connection can be established between this node and at least one node that precedes it in the sequence, such as... Figure 3 As shown by the dashed line, node 0 is connected to node 1, and node 7 is connected to node 8.

[0107] Similarly, when a second type of node exists, a connection is established between the second type of node and at least one node ordered after it.

[0108] The second type of node is a node that has not established a connection with any node that is ordered after it. Therefore, this type of node has no output. Figure 3 Node 10 is shown. For this type of node, a connection can be established between this node and at least one node ordered after it, such as... Figure 3 As shown by the dashed line, node 10 and node 11 are connected.

[0109] By applying the method of this application embodiment, nodes with only output and no input or nodes with only input and no output can be connected, thereby avoiding the situation where a node has no input or output, ensuring the normal structure of the ultrasound image segmentation model, and avoiding the impact of node connections on the operation of the ultrasound image segmentation model.

[0110] In one possible implementation, pooling and unpooling operations exist in the image segmentation network, which changes the size of the image feature map. Therefore, size requirements need to be set for feature fusion across different layers. For example, to ensure fusion efficiency across different layers, all convolutional kernels can be set to have the same size and number. Here, feature fusion across different layers requires size correspondence. In one example, when the image segmentation network is like... Figure 3 As shown, given nodes 0 to 12, the size of each node can be set using the following formula:

[0111]

[0112] Where C represents the number of channels in the input image; H represents the height of the input image; W represents the width of the input image; i represents the node number; and N represents the number of nodes.

[0113] To ensure that the feature map of each node conforms to the size requirements of that node, a size check can be set at even-numbered nodes. If the input feature map size does not match, it needs to be forcibly converted to the same size. Figure 3 In the example shown, the dimensions of nodes 0 and 12 are required to be [C, W, H], and the dimensions of nodes 2 and 10 are required to be [C, W, H]. The dimensions of nodes 4 and 8 are required to be: The size requirement for node 6 is as follows:

[0114] In one example, a method for illustrating an embodiment of this application is provided, such as... Figure 4 As shown, Figure 3 The image size transformation process between nodes connected to node 3 in the image segmentation network model shown is illustrated. Figure 4 As shown, the original image input size is [C, W, H]. When the image is output from node 0 to node 1, no pooling occurs, so the size of node 1 is also [C, W, H]. Although node 2 is not a pooling result from node 1, because the image size differs from the size requirement of this node, it needs to be forcibly pooled to meet the requirements. Image size requirements. Input the image features from nodes 0, 1, and 2 into node 3, but node 3 requires the image feature size to be [specific value missing]. Since the image feature size output by node 0 is [C, W, H], pooling is needed to force its output image feature size to be converted to [C, W, H]. The image feature sizes output by nodes 1 and 2 meet the input requirements of node 3, therefore no adjustment is needed. The image features from nodes 0, 1, and 2 are summed and then input into node 3. Since node 3 performs convolution and pooling operations, the feature map output size of node 3 is... The input size requirement for node 6 is: Then, the output of node 3 needs to undergo further pooling to meet the size requirements of node 6, while the input size requirement of node 9 is... Then, a deconvolution operation needs to be performed on the output of node 3 to meet the size requirements of node 9. Here, Pooling*2 represents the pooling operation of downsampling 2, and Unpooling*2 represents the unpooling operation of upsampling 2.

[0115] By applying the method of this application embodiment, and by setting corresponding size requirements for each node, and ensuring that the sizes of the corresponding nodes are the same, it is easier to process the image feature map, thereby accelerating the running speed of the ultrasound image segmentation model.

[0116] In one possible implementation, the boundaries of certain regions in human tissue are blurred, such as the cartilage region of the knee joint. Different doctors may have different ways of dividing the tissue, which can interfere with the relevant segmentation network model. Therefore, step S103 can be achieved through the following steps:

[0117] The output of the (N-1) / 2th node of the image segmentation network is connected to the input of the tolerance generation network, and the output of the tolerance generation network is connected to the output of the image segmentation network to obtain the ultrasound image segmentation model to be trained.

[0118] The tolerance region map is obtained by processing the bottom feature map output by the image segmentation network through the tolerance generation network. Therefore, the output of the image segmentation network can be concatenated with the input of the tolerance generation network. The ultrasound image segmentation model consists of an image segmentation network and a tolerance generation network. Combining the outputs of the image segmentation network and the tolerance generation network yields the final image segmentation result. Therefore, it is necessary to concatenate the output of the tolerance generation network with the output of the image segmentation network.

[0119] When connecting the output of the image segmentation network to the input of the tolerance generation network, the output of the (N-1) / 2th node of the image segmentation network can be connected to the input of the tolerance generation network, such as... Figure 5 As shown, Figure 3 The output of node 6 of the image segmentation network shown is connected to the input of the tolerance generation network. The sample ultrasound image is processed by the image segmentation network to obtain a probability map, which is then combined with the output of the tolerance generation network to obtain the final image segmentation result with tolerance regions.

[0120] In practical applications, when segmenting some ultrasound images, such as in certain auxiliary diagnostic tasks for conditions like knee arthritis, it is only necessary to delineate the main region. The segmentation boundary is not the focus of segmentation. Therefore, by applying the method of this application embodiment, a tolerance-based generative network can be added to blur the boundary region, avoiding the impact of different boundary divisions on the output results of the segmentation network model.

[0121] In one possible implementation, step S104 can be achieved through the following steps:

[0122] Step 1: Input the sample ultrasound image into the ultrasound image segmentation model to obtain the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network; wherein, the sample ultrasound image has a corresponding segmentation label, and the segmentation label represents the true value of the sample ultrasound image segmentation result.

[0123] In practical applications, the predicted tolerance map can be obtained through normalization to facilitate subsequent calculations. For example, the tolerance generation network can first process the probability map output by the nodes in the image segmentation network using the following formula:

[0124] P′=abs(0.5-P) (1)

[0125] Where P represents the probability map output by a node in the image segmentation network (e.g., the (N-1) / 2th node); ans is the absolute value function; and P′ represents the processed prediction tolerance map.

[0126] To facilitate observation of the segmentation results, P′ can be further normalized, setting a minimum value of 0 and a maximum value of 1. In one example, this can be done on all pixels in the P′ image as follows: subtract the minimum value of all pixels in the image from the value of each pixel, and then divide by the difference between the maximum and minimum values. This transforms P′ into a normalized image. Figure 5 The probability graph shown is represented by an edge region that is close to 0 (black region) and the rest that is close to 1 (white region).

[0127] The predicted tolerance map can be computed from the output of the (N-1) / 2th node in the image segmentation network. For example, such as... Figure 6 As shown, Figure 3 The image feature map generated by node 6 in the image segmentation network shown is used to generate an input probability map through a 1×1 convolution. The size of this input probability map is... The input probability map is then depooled to restore it to its original size. Finally, it is transformed and calculated using formula (1) and normalized to obtain the prediction tolerance map. Here, 1×1conv represents a 1×1 convolution operation, and Unpooling*8 represents an upsampling of 8 depooling operation.

[0128] Step 2: Multiply the elements at the same position in the prediction probability map and the prediction tolerance map to obtain the prediction segmentation result.

[0129] In one example, taking the prediction probability map and the prediction tolerance map as having dimensions [1, W, H], the elements at the same position are multiplied, that is, the product of the element (1, a, b) in the prediction probability map and the element (1, a, b) in the prediction tolerance map are calculated respectively, where a belongs to W and b belongs to H.

[0130] Step 3: Calculate the segmentation label of the sample ultrasound image and the loss of the predicted segmentation result.

[0131] In one example, the predicted segmentation result can be obtained by bitwise multiplying the matrix representing the predicted probability map with the matrix representing the predicted tolerance map. The segmentation label of the sample ultrasound image represents the true value of the segmentation result and can be represented as a labeled map. By calculating the error between the predicted segmentation result and the labeled map, the segmentation label of the sample ultrasound image and the loss of the predicted segmentation result are obtained.

[0132] Step 4: Adjust the parameters of the ultrasound image segmentation model based on the loss.

[0133] By applying the method of this application embodiment, the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network can be obtained respectively. The predicted segmentation result is obtained by combining the predicted probability map and the predicted tolerance map. The ultrasound image segmentation model is trained based on the loss between the predicted result and the ground truth, thereby avoiding the impact of different boundary divisions on the output result of the segmentation network model.

[0134] In one possible implementation, the method of this application embodiment may further include the following steps:

[0135] Step A: Adjust the connection weights between connected nodes in the image segmentation network based on the loss.

[0136] In practical applications, the connections between nodes are weighted connections. The weights of the connections between nodes can be the same or different, such as... Figure 7 As shown, the weighted connections between nodes have ω0 to ω n When connecting nodes, connection weights are randomly assigned to each connection, and the weights are adjusted after the loss is obtained.

[0137] Step B: For any connection, disconnect the connection if the connection weight is less than a preset weight threshold.

[0138] The preset weight threshold can be determined according to actual needs. The smaller the preset weight threshold, the higher the accuracy of the trained ultrasound image segmentation model. When the connection weight is less than the preset weight threshold, it is considered that the connection has a small impact on the segmentation result and can be ignored. In this case, the connection is disconnected. For example, during training, whenever a complete sample ultrasound image dataset passes through the ultrasound image segmentation model once and returns, connections with weights less than the preset weight threshold are disconnected. Figure 7 When training the ultrasound image segmentation model, if ω0 is less than the preset weight threshold, the corresponding node connection is disconnected; if ω3 is not less than the preset weight threshold, the corresponding node connection is retained.

[0139] By applying the method of this application embodiment, the connection weights between nodes are judged, and the connections with connection weights less than a preset weight threshold are disconnected. This reduces the number of node connections that are less relevant to the output results of the ultrasound image segmentation model, allowing the ultrasound image segmentation model to dynamically adapt to the combination of high and low-dimensional features during training. This removes redundant features with low connection weights and selects effective cross-layer features with high connection weights, thereby improving the accuracy of the ultrasound image segmentation model and accelerating its operation.

[0140] In a second aspect of this application, an ultrasound imaging detection method is provided, the method comprising: Figure 8 The steps shown are as follows:

[0141] Step S801: Obtain the image to be detected.

[0142] Step S802: Input the image to be detected into the image segmentation network to obtain the segmentation result, wherein the image segmentation network is trained by any of the methods in the first aspect of the embodiments of this application.

[0143] By applying the method of the embodiments of this application, the image to be detected can be input into the image segmentation network trained by any of the methods of the first aspect of the embodiments of this application, thereby achieving the segmentation result and realizing the automated segmentation of ultrasound images, avoiding manual segmentation judgment and reducing the cost of ultrasound image segmentation.

[0144] In a third aspect of this application, an ultrasound image segmentation model training device is provided, the device comprising: Figure 9 Results shown:

[0145] The parameter acquisition module 901 is used to acquire the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1.

[0146] The node connection module 902 is used to take N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes in the image segmentation network, and disconnect or reconnect the connections between nodes with a probability of P to generate the image segmentation network.

[0147] The network connection module 903 is used to obtain the tolerance generation network and connect the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained.

[0148] Model training module 904 is used to train the ultrasound image segmentation model using sample ultrasound images.

[0149] The apparatus of this application embodiment can generate an image segmentation network by disconnecting or reconnecting the connections between nodes with a probability of P. The image segmentation network is then connected to a tolerance generation network to generate an ultrasound image segmentation model. The ultrasound image segmentation model can be trained and used for ultrasound image segmentation, thereby achieving automated segmentation and detection of ultrasound images without manual segmentation and reducing the cost of ultrasound image segmentation.

[0150] In one possible implementation, the node connection module includes:

[0151] The node generation submodule is specifically used to generate N nodes, where the i-th node includes a convolutional layer, the (i+1)-th node includes a convolutional layer and a pooling layer, and i is an even number belonging to 0 to N-1;

[0152] The node connection submodule is specifically used to establish connections between each node and the other K nodes.

[0153] The connection determination submodule is specifically used to determine whether to disconnect nodes that are connected with a probability of P, and to determine whether to connect nodes that are not connected with a probability of P. The result of the determination is used to obtain the image segmentation network.

[0154] The apparatus of this application embodiment can randomly disconnect or connect nodes of a regular network based on probability P, thereby generating a small-world network in this application embodiment. This enables the generation of new networks guided by the small-world theory of complex networks, thereby reducing the complexity of ultrasound image segmentation model design, improving the flexibility of convolutional combination, enhancing the feature representation ability of ultrasound image segmentation model, and thus improving the accuracy of ultrasound image segmentation model.

[0155] In one possible implementation, the node connects the sub-module, including:

[0156] The node connection unit is specifically used to establish connections between each node and its K nearest neighbors.

[0157] In one possible implementation, the device further includes:

[0158] The first type of node connection module is used to establish a connection between a first type of node and at least one node that is ordered before it when a first type of node exists, wherein the first type of node is a node that has not established a connection with any node ordered before it.

[0159] The apparatus of this application embodiment can connect nodes that have only outputs and no inputs, as well as nodes that have only inputs and no outputs, thereby avoiding situations where nodes have no inputs or outputs, ensuring the normal structure of the ultrasound image segmentation model, and preventing the operation of the ultrasound image segmentation model from being affected by the connection between nodes.

[0160] In one possible implementation, the size of the i-th node is:

[0161]

[0162] The apparatus of this application, by setting corresponding size requirements for each node and ensuring that the sizes of the corresponding nodes are the same, facilitates the processing of image feature maps, thereby accelerating the running speed of the ultrasound image segmentation model.

[0163] In one possible implementation, the network connectivity module includes:

[0164] The network connection submodule is specifically used to connect the output of the (N-1) / 2th node of the image segmentation network with the input of the tolerance generation network, and to connect the output of the tolerance generation network with the output of the image segmentation network to obtain the ultrasound image segmentation model to be trained.

[0165] The apparatus of this application embodiment can blur the boundary region by adding a tolerance generation network, thereby avoiding the impact of different boundary divisions on the output results of the segmentation network model.

[0166] In one possible implementation, the model training module includes:

[0167] The sample ultrasound image input submodule is specifically used to input the sample ultrasound image into the ultrasound image segmentation model to obtain the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network. The sample ultrasound image has a corresponding segmentation label, which represents the true value of the sample ultrasound image segmentation result.

[0168] The product operation submodule is specifically used to perform product operations on elements at the same position in the prediction probability map and the prediction tolerance map to obtain the prediction segmentation result;

[0169] The loss calculation submodule is specifically used to calculate the segmentation label of the sample ultrasound image and the loss of the predicted segmentation result;

[0170] The parameter adjustment submodule is specifically used to adjust the parameters of the ultrasound image segmentation model based on the loss.

[0171] The apparatus of this application embodiment can acquire the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network, respectively. The predicted segmentation result is obtained by combining the predicted probability map and the predicted tolerance map. The ultrasound image segmentation model is trained based on the loss between the predicted result and the ground truth, thereby avoiding the impact of different boundary divisions on the output result of the segmentation network model.

[0172] In one possible implementation, the device further includes:

[0173] The weight adjustment module is used to adjust the connection weights between connected nodes in the image segmentation network based on the loss.

[0174] The connection disconnection module is used to disconnect any connection if the connection weight of that connection is less than a preset weight threshold.

[0175] The apparatus of this application determines the connection weights between nodes and disconnects connections with weights less than a preset weight threshold, thereby reducing node connections with low correlation to the output of the ultrasound image segmentation model. This allows the ultrasound image segmentation model to dynamically adapt to the combination of high and low-dimensional features during training. This process removes redundant features with low connection weights and selects effective cross-layer features with high connection weights, improving the accuracy of the ultrasound image segmentation model and accelerating its operation.

[0176] In a fourth aspect of this application, an ultrasound imaging detection device is provided, the device comprising: Figure 10 The structure shown is as follows:

[0177] The image acquisition module 1001 is used to acquire the image to be detected.

[0178] The ultrasound image detection module 1002 is used to input the image to be detected into an image segmentation network to obtain a segmentation result, wherein the image segmentation network is trained by any of the methods in the first aspect of the embodiments of this application.

[0179] The apparatus of this application embodiment can input the image to be detected into the image segmentation network trained by any of the methods in the first aspect of this application embodiment, thereby segmenting the image and achieving automated segmentation of ultrasound images, avoiding manual segmentation judgment and reducing the cost of ultrasound image segmentation.

[0180] This application also provides an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104.

[0181] Memory 1103 is used to store computer programs;

[0182] When processor 1101 executes the program stored in memory 1103, it performs the following steps:

[0183] Obtain the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1;

[0184] N is used as the number of nodes in the image segmentation network, K is used as the number of connections between each node and other nodes in the image segmentation network, and the connections between nodes are disconnected or reconnected with probability P to generate the image segmentation network.

[0185] Obtain the tolerance generation network, and connect the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained;

[0186] The ultrasound image segmentation model was trained using sample ultrasound images.

[0187] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0188] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0189] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0190] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0191] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described ultrasound image segmentation model training methods.

[0192] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the ultrasound image segmentation model training methods described above.

[0193] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0194] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0195] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0196] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for training an ultrasound image segmentation model, characterized in that, The method includes: Obtain the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1; N is the number of nodes in the image segmentation network, K is the number of connections between each node and other nodes in the image segmentation network, and the connection between each node is disconnected or reconnected with probability P to generate the image segmentation network. A tolerance generation network is obtained, and the image segmentation network is connected to the tolerance generation network to obtain an ultrasound image segmentation model to be trained; the tolerance generation network is used to output a tolerance map. The ultrasound image segmentation model is trained using sample ultrasound images.

2. The method according to claim 1, characterized in that, The process of generating the image segmentation network by using N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes, and disconnecting or reconnecting the connections between nodes with probability P, includes: Generate N nodes, where the i-th node includes a convolutional layer, the (i+1)-th node includes a convolutional layer and a pooling layer, and i is an even number between 0 and N-1; For each node, establish connections between that node and the other K nodes; The image segmentation network is obtained by determining whether to disconnect nodes with a probability of P and whether to connect nodes without a probability of P, and by performing the determination results.

3. The method according to claim 2, characterized in that, For each node, establishing connections between that node and the other K nodes includes: For each node, a connection is established between that node and its K nearest neighbors.

4. The method according to claim 2, characterized in that, The method further includes: When a first type of node exists, a connection is established between the first type of node and at least one node that is ranked before it, wherein the first type of node is a node that has not established a connection with any node that is ranked before it.

5. The method according to claim 2, characterized in that, The size of the i-th node is: ； Where C represents the number of channels in the input image; H represents the height of the input image; W represents the width of the input image; i represents the node number; and N represents the number of nodes.

6. The method according to claim 1, characterized in that, The step of connecting the image segmentation network with the tolerance generation network to obtain the ultrasound image segmentation model to be trained includes: The output of the (N-1) / 2th node of the image segmentation network is connected to the input of the tolerance generation network, and the output of the tolerance generation network is connected to the output of the image segmentation network to obtain the ultrasound image segmentation model to be trained.

7. The method according to claim 1, characterized in that, The training of the ultrasound image segmentation model using sample ultrasound images includes: The sample ultrasound image is input into the ultrasound image segmentation model to obtain the predicted probability map output by the image segmentation network and the predicted tolerance map output by the tolerance generation network; wherein, the sample ultrasound image has a corresponding segmentation label, and the segmentation label represents the ground truth value of the sample ultrasound image segmentation result. The predicted segmentation result is obtained by multiplying the elements at the same position in the predicted probability map and the predicted tolerance map. Calculate the segmentation label of the sample ultrasound image and the loss of the predicted segmentation result; The parameters of the ultrasound image segmentation model are adjusted based on the loss.

8. The method according to claim 7, characterized in that, The method further includes: Based on the loss, adjust the connection weights between connected nodes in the image segmentation network; For any given connection, disconnect the connection if the connection weight is less than a preset weight threshold.

9. An ultrasound imaging detection method, characterized in that, The method includes: Acquire the image to be detected; The image to be detected is input into an image segmentation network to obtain a segmentation result, wherein the image segmentation network is trained by any one of the methods described in claims 1-8.

10. A training device for an ultrasound image segmentation model, characterized in that, The device includes: The parameter acquisition module is used to obtain the number of nodes N, the number of connections K, and the connection probability P, where N is an odd number greater than 9 and K is an integer greater than 1. The node connection module is used to take N as the number of nodes in the image segmentation network, K as the number of connections between each node and other nodes in the image segmentation network, and disconnect or reconnect the connections between nodes with a probability of P to generate the image segmentation network. A network connection module is used to acquire a tolerance generation network, connect the image segmentation network to the tolerance generation network to obtain an ultrasound image segmentation model to be trained; the tolerance generation network is used to output a tolerance map. The model training module is used to train the ultrasound image segmentation model using sample ultrasound images.

11. An ultrasonic imaging detection device, characterized in that, The device includes: The image acquisition module is used to acquire images to be detected. An ultrasound image detection module is used to input the image to be detected into an image segmentation network to obtain a segmentation result, wherein the image segmentation network is trained by the method described in any one of claims 1-8.

12. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-9.

Images