Biological tissue identification and processing method based on task consistency

Abstract

The invention relates to a biological tissue identification and processing method based on task consistency, comprising: constructing a network model based on a unified task perception consistency interaction framework, the network model comprising a skeleton network for extracting multi-scale image features, a main branch and at least two additional branches; wherein the skeleton network is based on an encoder-decoder architecture of a deep learning model; the main branch is in parallel connection with the at least two additional branches, and tasks executed by the additional branches have strong correlation and weak correlation with the main task respectively; training sample data is used to train the network model, and a biological tissue identification processing training model is obtained; and identifying and processing the biological tissue image data by using the biological tissue identification processing training model. According to the method, the dynamic capability of deep learning is enhanced, supervised and semi-supervised scenes are benefited, and the method has high effectiveness and robustness in the aspect of coping with complex challenges of medical image segmentation.

Description

Technical Field

[0001] The present invention relates to a biological tissue recognition and processing method based on task consistency, and specifically to a semi-supervised image recognition and segmentation method based on a unified task perception consistency interaction framework, which belongs to the technical field of image pattern recognition based on artificial neural networks and the technical field of artificial intelligence applied to medicine. Background Art

[0002] In in vitro fertilization technology, fertilized cells need to be cultured until the seventh day to form an embryo. At this time, the implantation potential needs to be evaluated based on the morphology of various tissues within the embryo. Currently, this tissue morphology analysis mainly relies on the subjective judgment of the operator.

[0003] In the prior art, exploration has begun on automatic identification and evaluation of different embryonic tissue morphologies to provide a more objective evaluation; for example, Chinese patent application CN202210978474.8 proposes a multi-stage intelligent analysis system based on embryonic development images, which uses a YOLO network including 24 convolutional layers, 4 pooling layers, and 2 fully connected layers to output developmental stage category labels for blastocyst tissue images to achieve multi-stage identification and segmentation, but it can only segment the entire blastocyst image from the culture dish background, and cannot achieve the identification and segmentation of the internal structure image of the blastocyst tissue. For another example, in the prior art, traditional methods, such as the level set method, have been used to segment the inner cell mass and trophoblast of the blastocyst image; with the advancement of deep learning, some studies have attempted to combine traditional methods with deep learning for the identification of the internal structure of the blastocyst tissue in blastocyst images. For example, traditional features are extracted and input into a two-layer neural network to identify the vitelline membrane (ZP), trophoblast (TE), and inner cell mass (ICM) in blastocyst images, and a fully convolutional network (FCN) is used to further segment the inner cell mass (ICM). However, these data-driven networks usually require a large amount of labeled data for network training, but obtaining labeled data in clinical practice consumes time and a lot of resources, which hinders the widespread application of deep learning technology.

[0004] In the prior art, methods that exploit semi-supervised learning by minimizing the entropy of predicted probabilities have also begun to be explored to be applicable to unlabeled data. For example, adversarial training is used to regularize the segmentation model, aiming to minimize the difference between the segmentation distributions of labeled and unlabeled images; interpolation consistency (IC) is used to apply consistency between different versions of the input data; a regularized discard network (RD) is used to enforce consistent segmentation between the discarded information of the network; a teacher-student framework is used to enforce prediction consistency between labeled and unlabeled samples during the SSL process, and Monte Carlo discard is combined to estimate the uncertainty in the teacher model prediction; an aggregation and decoupling framework is used to capture distribution-invariant features; a method that uses global information in segmentation masks to estimate segmentation uncertainty, etc. However, these methods focus on pixel-level consistency without fully exploring the underlying representation consistency between different feature spaces.

[0005] According to existing experience, how to effectively use unlabeled data to identify different problems by combining embryo skeleton and boundary features is a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a biological tissue recognition and processing method based on task consistency, comprising the steps of: constructing a network model based on a unified task-aware consistency interaction framework, the network model comprising a skeleton network for extracting multi-scale image features, a main branch and at least two additional branches; wherein the skeleton network is based on an encoder-decoder architecture of a deep learning model, and the output result of the skeleton network is input into the main branch and at least two additional branches; the main branch and the at least two additional branches are convolutional neural networks in a parallel relationship, wherein the main branch is used to perform a main task, the task performed by at least one additional branch has a strong correlation with the main task, and the task performed by at least another additional branch has a weak correlation with the main task; obtaining training sample data, the training sample data being biological tissue image data, a part of the biological tissue image data being labeled data for supervised training, and the other part being unlabeled data for semi-supervised training; using the training sample data to train the network model based on the unified task-aware consistency interaction framework to obtain a biological tissue recognition and processing training model; and using the biological tissue recognition and processing training model to recognize and process the biological tissue image data.

[0007] In the above technical scheme, the main branch is a segmentation branch, which is used to generate a pixel-level probability map and generate a pixel-level output result; an additional branch is a level set branch, which is used to generate a target level set representation and generate a geometric level output result, which has a strong correlation with the main task; another additional branch is a point set branch, which is used to synthesize a point representation related to the target structure and generate a point-level output result, which has a weak correlation with the main task.

[0008] In the above technical solution, the segmentation branch performs supervised training, and the level set branch and the point set branch perform supervised training and semi-supervised training simultaneously.

[0009] In the above technical solution, the biological tissue image data has dimensional structure, wherein C represents the number of color channels of the biological tissue image, H and W represent the height and width of the biological tissue image respectively, and R represents the real number space; the biological tissue image data is processed by the skeleton network to obtain feature sets of different scales Indicates that, the dimension , and Corresponding to the use of The downsampling factor of C, H, and W is processed; the integer D indicates the degree of downsampling performed at each stage.

[0010] In the above technical solution, the network model based on the unified task-aware consistency interaction framework adopts the loss function when training with labeled data for:

[0011] in, is the loss function for supervised training of the segmentation branch, is the loss function for supervised training of the point set branch, is the loss function for supervised training of the level set branch, is the strong and weak task-aware consistency function, , , , is the weight coefficient.

[0012] In the above technical solution, the network model based on the unified task-aware consistency interaction framework adopts the loss function of unlabeled data training for:

[0013] in, is the loss function for unsupervised training of the point set branch, is the loss function for unsupervised training of the level set branch, is the strong and weak task-aware consistency function, , , is the weight coefficient.

[0014] In the above technical solution, the loss function of the network model based on the unified task-aware consistency interaction framework for semi-supervised learning is The overall expression is:

[0015] in, , is the weight coefficient, The loss function used for training with labeled data is used for the network model based on the unified task-aware consistency interaction framework. The loss function used when training with unlabeled data is adopted for network models based on a unified task-aware consistency interaction framework.

[0016] In the above technical solution, using the training sample data to train the network model based on the unified task-aware consistency interaction framework includes: Step S100: Randomly initialize segmentation branch parameters , point set branch parameter , level set branch parameter ; Step S200: When the predetermined loss expectation or training times are not completed, execute: Step S210: Obtain each batch of training samples ;in, They are labeled datasets The corresponding input image and label map; For unlabeled dataset The corresponding input image; Step S220: Calculate the prediction results of the segmentation branch, point set branch, and level set branch respectively ; Step S230: Utilize Transformation formula through point set branching generate , through the transformation formula of the level set branch generate ; Step S240: Prediction results By formula Generate point set representation features ; Step S250: Prediction results By formula Generate level set representation features ; Step S260: Prediction results By formula Generate pixel representation features ; Step S270: Prediction results By formula Generate point set representation features ; Step S280: Obtaining the overall loss ; Step S290: Update model parameters by back propagation ; Step S300: When the predetermined loss expectation or training times are completed, the loop ends; Step S400: Return the obtained segmentation branch parameters , point set branch parameter , level set branch parameter .

[0017] The present invention also provides a biological tissue recognition and processing device based on task consistency, comprising: a model construction module, used to construct a network model based on a unified task-aware consistency interaction framework, the network model comprising a skeleton network for extracting multi-scale image features and a main branch and at least two additional branches; wherein the skeleton network is based on an encoder-decoder architecture of a deep learning model, and the output result of the skeleton network is input to the main branch and at least two additional branches; the main branch and the at least two additional branches are convolutional neural networks in parallel, wherein the main branch is used to perform a main task, the task performed by at least one additional branch has a strong correlation with the main task, and the task performed by at least another additional branch has a weak correlation with the main task; a training sample acquisition module, used to acquire training sample data, the training sample data being biological tissue image data , part of the biological tissue image data is labeled data for supervised training, and the other part is unlabeled data for semi-supervised training; a model training module, using the training sample data to train the network model based on the unified task-aware consistency interaction framework to obtain a biological tissue recognition processing training model; wherein, after the biological tissue image data in the training sample data is input into the skeleton network, feature sets of different scales are obtained, and the feature sets are respectively input into the main branch and at least two additional branches for training, and the main branch and at least two additional branches have loss functions corresponding to the tasks they perform, and these loss functions are compared and guided by the information between related tasks to utilize the synergy between these diverse tasks; a data processing module, using the biological tissue recognition processing training model to identify and process the biological tissue image data.

[0018] The present invention has achieved the following technical effects: Based on the derivation results of different task-aware consistency, a new unified task-aware consistency interaction (UniTask+) framework is developed, which fully exploits the synergy between these diverse tasks and promotes the effective use of task-aware knowledge to improve the performance of semi-supervised medical image segmentation. The UniTask+ framework method promotes the exploration of intrinsic segmentation perturbations at three key levels (pixel level, geometric level, and point level), enhances learning dynamics, and benefits both supervised and semi-supervised scenarios. Experimental evaluations on ICM, blastocyst, and left atrium datasets have reached the state-of-the-art level, and are effective and robust in addressing the complex challenges of medical image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a correlation diagram for performing three tasks on a blastocyst image; Figure 2 This is the structure diagram of the unified task-aware consistency interaction (UniTask+) framework; Figure 3 The network structure diagram for the segmentation branch; Figure 4 This is the network structure diagram of the point set branch and the level set branch. DETAILED DESCRIPTION

[0020] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0021] like Figure 1 As shown in the figure, factors such as the fullness and proportion of the inner cell mass (ICM) represented by the blastocyst image are key evaluation indicators for determining whether the embryo can continue to develop better; at the same time, other structural morphologies of the blastocyst stage, such as the blastocyst cavity, trophoblast and zona pellucida, are also of great significance. Therefore, it is necessary to effectively combine the task of identifying structural information and the task of identifying boundary information with the traditional segmentation task to achieve automatic segmentation of different blastocyst tissues; and in this case, it is also necessary to make full use of unlabeled data. Based on this requirement, the segmentation task of blastocyst images needs to include at least the following main contents: (1) Point set representation task, which is used to represent the structural information of the inner cell mass (ICM). Figure 1 The skeleton point set depicted by the point set representation task in effectively captures and represents the overall distribution trend of the inner cell mass (ICM) at the point level; however, this representation does not include boundary information. (2) Semantic segmentation task; its results show that the clustering in the blastocyst image exhibits significant irregular patterns at the pixel level. (3) Distance transformation task, which is used to represent the boundary information of the inner cell mass (ICM). There is an intrinsic connection between the output results obtained from these three tasks. Specifically, Figure 1As shown, the clustering of the inner cell mass (ICM) in the blastocyst image shows a significant irregular pattern at the pixel level; there is a weak correlation between the output results of the point set representation task and the semantic segmentation task, and a strong correlation between the output results of the semantic segmentation task and the distance conversion task.

[0022] Therefore, fusing and superimposing the output results of the point set representation task, the semantic segmentation task, and the distance conversion task can take into account both the weak correlation and the strong correlation mentioned above, such as Figure 1 As shown in Figure 2, through the semantic segmentation task as an intermediary, the geometric information extracted from the distance conversion task can be converted into point representation and connected with the point-level task obtained from the point set representation task. Therefore, it is necessary to consider the shared basic features and similar processing pipelines between these tasks within a network framework to achieve Figure 1 Task perception consistency among the three tasks shown.

[0023] Based on the task-aware smoothness (TAS) hypothesis, it is assumed that the mutual information between the output results of the above three tasks remains unchanged during the task conversion process, including strong correlation (i.e. strong task-aware consistency) and strong-to-weak correlation (i.e. strong-to-weak task-aware consistency). This stable exchange of information promotes feature consistency and effectively minimizes the difference between the learned features and their intrinsic properties, which is particularly suitable for processing unlabeled data.

[0024] The mathematical definition of the task-aware smoothness hypothesis is: given an input , and two outputs , Two different tasks A and B, in the Applying Task Transformation Afterwards, the corresponding results and Should be the same.

[0025] When task switching The task-aware smoothness hypothesis is bidirectional, with task B being defined as being strongly correlated with task A and vice versa. is unidirectional, and task B is called a weakly related task. Finally, the task-aware consistency is obtained by minimizing: (1) in, is the metric function; and Indicates that tasks A and B are executed and share the same backbone network; represents the conditional probability of the model. When B is a weakly related task, the corresponding weak task-aware consistency is the same as formula (1). This equation can effectively cover many previous studies using the dual-anchor two-stream architecture in semi-supervised learning. These studies have utilized two independent branches in their methodology to deal with SSL tasks, usually one branch is used to process labeled data and the other branch is used to utilize unlabeled data or combine labeled and unlabeled information in a synergistic manner. The equation in question may reflect the core principles or mathematical formulas of these methods to deal with consistency specification, feature extraction, or joint learning between different branches to improve model performance under limited supervision.

[0026] In addition, when B is a strongly related task, the corresponding strong task consistency is: (2) in, yes This method helps to integrate and interact multiple levels of task-aware consistency in a unified semi-supervised learning framework.

[0027] For the above formula (2), if strongly related tasks A and weakly related tasks C can be found in the main segmentation task B, this consistency can be achieved: (3) in, It is a composite transformation from strongly related tasks to weakly related tasks, which can also be expressed as , followed by a one-way transformation Continuous transformation of represents the performance task C. By leveraging this full task-aware consistency in Eq. (3), we can potentially improve the generalization ability and accuracy of the main segmentation task B by learning richer and more discriminative feature representations from locally and globally related auxiliary tasks.

[0028] In order to solve the above technical problems, the present invention proposes a task-based perspective to identify and segment blastocyst images through semi-supervised learning, focusing on the task invariance and task-specific dependency of segmentation. Based on the derivation result that different task perceptions have consistency, the present invention develops a new unified task-aware consistency interaction (UniTask+) framework, which includes a main branch based on a skeleton network to implement medical segmentation (MS), and two additional branches based on the same skeleton network to perform strong / weakly related tasks; among them, a contour / level set (LS) branch focuses on extracting and processing geometric information to promote strong consistency; and a point set (PS) branch is used to generate point set representations to stimulate weak consistency under the underlying task perturbations.

[0029] More specifically, the present invention provides a method for biological tissue recognition and processing based on task consistency, comprising the steps of: constructing a network model based on a unified task-aware consistency interaction framework, the network model comprising a skeleton network for extracting multi-scale image features, a main branch and at least two additional branches; wherein the skeleton network is based on an encoder-decoder architecture of a deep learning model, and the output result of the skeleton network is input into the main branch and at least two additional branches; a convolutional neural network in which the main branch and the at least two additional branches are in a parallel relationship, wherein the main branch is used to perform a main task, the task performed by at least one additional branch has a strong correlation with the main task, and the task performed by at least another additional branch has a weak correlation with the main task; obtaining training sample data, the training sample data being biological tissue image data, a part of the biological tissue image data being labeled data for supervised training, and the other part being unlabeled data for semi-supervised training; using the training sample data to train the network model based on the unified task-aware consistency interaction framework to obtain a biological tissue recognition and processing training model; and using the biological tissue recognition and processing training model to recognize and process the biological tissue image data.

[0030] Preferably, the biological tissue image data in the training sample data is input into the skeleton network to obtain feature sets of different scales, which are respectively input into the main branch and at least two additional branches for training. The main branch and at least two additional branches have loss functions corresponding to the tasks they perform respectively, and these loss functions are compared and guided by the information between related tasks to utilize the synergy between these diverse tasks.

[0031] Preferably, the unified task-aware consistency interaction (UniTask+) framework provided by the present invention is as follows: Figure 2 As shown in the figure, it includes a skeleton network and three branches based on the skeleton network, namely: (i) medical segmentation branch (MS), which is mainly used to generate pixel-level probability maps; (ii) level set branch (LS), which is mainly used to generate target level set representations; (iii) point set branch (PS), which is mainly used to synthesize point representations related to the target structure. The blastocyst image X is input into the skeleton network, and feature maps of different scales are obtained after passing through the skeleton network. , and enter the three branches of MS, LS, and PS accordingly to generate pixel-level, geometric-level, and point-level output results, and the loss function is guided by the comparison of information between the main task and the strongly / weakly related tasks. This coherent framework fully exploits the synergy between these diverse tasks and promotes the effective use of task-aware knowledge to improve the performance of semi-supervised medical image segmentation.

[0032] The skeleton network in the UniTask+ framework provided by the present invention is based on a typical encoder-decoder architecture, or in other words, the UniTask+ framework can be seamlessly integrated into any encoder-decoder architecture. Figure 2 As shown, it includes an input layer, an encoding layer / encoder, several hidden layers, a decoding layer / decoder, and one or more output layers. The encoder-decoder architecture is a deep learning model structure that is widely used in natural language processing, image processing, speech recognition and other fields; this structure can handle sequence-to-sequence tasks such as machine translation, text summarization, dialogue systems, sound conversion, etc. Among them, the role of the encoder is to receive the input sequence and convert it into a fixed-length context vector. The encoder can be any type of deep learning model; the goal of the decoder is to convert the context vector generated by the encoder into an output sequence. The decoder can also be various types of deep learning models, but usually the same type of model is used as the encoder to maintain consistency. When training a model based on the encoder-decoder architecture, the goal is to minimize the difference between the output sequence predicted by the model and the actual output sequence, which is usually achieved by calculating the loss function and using optimization algorithms such as backpropagation and gradient descent for parameter updates.

[0033] like Figure 2 As shown, for the main task, the MS branch obtains the segmentation result ; For strongly correlated tasks, the LS branch obtains a geometric representation under the contour function ; For weakly related tasks, the PS branch obtains the skeleton representation point set of the object When processing labeled data, the output of these three tasks—segmentation prediction ( ), contour lines represent ( ) and point set representation ( )-are compared with the corresponding true values ​​to evaluate their respective supervision losses. When dealing with unlabeled data, the level set reconstruction from the LS branch is used ( ) and the point set representation from the PS branch ( ) respectively calculate the unsupervised loss and use the segmentation prediction from the MS branch ( ) as a consistency anchor. This cross-task participation enhances the ability of the UniTask+ framework network to extract meaningful representations from semi-supervised learning scenarios, regardless of whether the data is labeled or unlabeled. By minimizing the differences between pixels and pixels, pixels and contours, pixels and skeleton points, and contours and points, the UniTask+ framework network can learn consistent features of pixel-level, geometric-level, and point-level data in three tasks, maintaining strong, weak, and strong-to-weak task-aware consistency, thereby updating the entire UniTask+ framework network, making full use of unlabeled data and improving the performance of the UniTask+ framework network for blastocyst image recognition and segmentation.

[0034] For the blastocyst image input to the skeleton network have The dimension structure of X, where C represents the number of channels. For RGB images, the number of color channels is usually C=3, while H and W represent the height and width of the image X, respectively. After a series of calculation processes in the backbone network, feature maps of different scales are generated in sequence. Indicates that the dimension and According to The downsampling factor is reduced, that is,

[0035] The integer D represents the degree of downsampling performed at each stage. Each such feature map There is a corresponding number of channels, recorded as After the multi-scale feature extraction of the skeleton network, the output of the skeleton network provides a final feature representation, called .

[0036] Multi-scale branches are introduced after the skeleton network to form the backbone network architecture. The segmentation branch network uses the feature set output by the skeleton network , the final recognition feature space is derived through a series of convolution operations Output have dimensional structure, where K represents the number of categories, and for binary segmentation tasks, K=2. The network structure and loss function of the segmentation branch are as follows Figure 3 As shown, its typical structure includes an input layer, several convolutional network layers and an output layer; optionally, the input layer of the segmentation branch can be merged / shared with an output layer of the skeleton network.

[0037] At the same time, a point set branch network is introduced after the skeleton network to obtain the blastocyst image The point set representation of the skeleton information explores the potential weak task-aware structure to capture the complex structural details of the skeleton. Specifically, the point set branch is as follows: Figure 4 As shown, the input of the point set branch is expressed as , that is, the feature map in the skeleton network It contains two learning heads: category head and location head. It reflects that each point in the data should carry both its category identification and relative spatial coordinates. Input features The initial mapping of produces an estimated point representation , which is represented by the object confidence score and relative position coordinate mapping The category head mainly identifies whether a given point belongs to the skeleton structure, while the location head is specifically used to estimate the positional relationship between each point and the center point of the image. In order to align with the true value, Introduced To convert, so as to get the actual point set representation , including goal scoring and relative position coordinate mapping .

[0038] Using transformation function The pixel-level segmentation mask Convert to a suitable point set representation The mathematical expression of this transformation is as follows: (4) Among them, U represents the union operation, and the symbol and Represent morphological erosion and opening operations respectively. is a predefined structural element, and N is determined to be the maximum value for which the erosion operation does not result in an empty set, that is, This iterative process essentially extracts a skeleton-like structure by repeatedly corroding and subtracting the corroded image from the original image, thereby maintaining the core shape and connectivity of the segmented object, represented as a point set. . Then the goal is scored , which can also be expressed as For a matrix with binary values ​​from . And we will Convert to Point Set , where a and b represent the horizontal and vertical relative distances between the current point and the center of the image, express The data points in .

[0039] In addition, by using the target score estimate and relative position coordinates The calculated joint loss introduces the overall optimization strategy of the PS branch. This approach plays an important role in preventing the misidentification between individual points and their spatial locations.

[0040] For some prediction points, there may be higher confidence scores , but their position estimates may deviate significantly. Therefore, effectively combining these two types of information becomes crucial to reduce ambiguity and disambiguate between different classes, which will ensure that the predicted points can not only be confidently classified but also correctly located, thus helping to improve the performance in identifying and partitioning complex structures in the skeleton. However, the results predicted by MS branches Information that cannot be generated from the PS branch alone The PS branch follows formula (1), so the PS branch is called a weakly related task.

[0041] The present invention associates the strongly correlated task assignment with a contour distance representation branch (i.e., level set branch) to capture the geometric active contours and distance information of the internal structure of the blastocyst tissue image, thereby obtaining information at the geometric level and pixel level, and minimizing these consistencies to further learn the boundaries, thereby increasing the consistency of the strongly correlated tasks. Here, the feature set output by the skeleton network is used As the input of the level set branch, it simulates the role of the level set task through the level set branch. Specifically, this is achieved through a 1×1 convolutional layer and normalized using the tanh function to obtain the final level set output ,like Figure 4 is shown in the level set branch of

[0042] As a strongly related task, and Should be bidirectional conversion, can be from arrive The transformation is set to the forward transformation , the transformation is defined as: (5) Among them, Inf represents the infimum; represents the L2 norm; , They are from , Two different pixels; is the zero level set, which also represents the contour of the target object; and They represent the inner and outer areas of the target object respectively.

[0043] Due to the non-differentiability of formula (5), it is difficult to solve its inverse mapping problem, the following smoothing function can be used as the inverse mapping: (6) After the input is amplified by k times, it is differentiable and reversible, which can increase the task perception consistency.

[0044] Based on the above, given a pair of supervised data , after applying the skeleton network to obtain features at different scales, the MS branch obtains the segmentation results , the LS branch obtains a geometric representation under the level set , the PS branch obtains the skeleton point representation of the object .

[0045] For the loss function of the main task, such as Figure 3 As shown in the loss function of the MS branch, under supervised data, directly use and To calculate the cross entropy to obtain

[0046] (7) in, Representation calculation and The cross entropy of .

[0047] For the loss of weakly related tasks, such as Figure 4 As shown in the loss function of the PS branch, the real label is first converted to Convert to (Including True Goal Score And the real location map ), and use and (including predicted goal score and predicted location map ), the loss is calculated by the PS branch Includes score loss and position information loss

[0048] (8) Therefore, we can get the loss function of point set representation (9) Wherein, μ1 and μ2 represent weight coefficients respectively, preferably, μ1=0.1 and μ2=1.0.

[0049] Considering the weak task loss consistency in formula (1), we use and ( It is through the formula ) to obtain the weak task consistency loss .

[0050] (10) The training goal is to ensure that the predicted Y is very close to the true value Y. Therefore, the weight assigned to each loss is set to 1. The overall loss of the PS branch is (11) For the loss function of strongly related tasks, such as Figure 4 As shown in the loss function of the LS branch, firstly, the formula generate , then use and generated by the LS branch To calculate the mean squared error (MSE) as the distance loss function: (12) Secondly, we can also predict the and through Generated Calculate the distance loss function:

[0051] (13) In addition, the LS branch generates Can be reversed get , so we can calculate this partial consistency loss : (14) in, for function.

[0052] The sum of equations (13) and (14) conforms to equation (2), which reflects the strong consistency goal of the task to be achieved by the present invention. Therefore, the overall level set branch (LS) loss can be defined as : (15) in, , , is the weight coefficient; preferably =0.01, =0.01 and =1.

[0053] like Figure 4 As shown in Figure 2, the strong and weak task-aware consistency in the UniTask+ framework is also introduced.

[0054] (16) In summary, the supervised loss of the UniTask+ framework Defined as: (17) in, , , , is the weight coefficient, preferably =1.0, =0.006, =1.0 and =0.001.

[0055] Different from the previous UniTask framework: (18) The process of semi-supervised learning of the UniTask+ framework of the present invention is as follows: Semi-supervised training with task-aware consistency via UniTask+: enter:

[0056] in, They are labeled datasets Corresponding input image and label map; For unlabeled dataset The corresponding input image.

[0057] Output: MS branch parameters , PS branch parameters , LS branch parameters (Note that they share the same trunk) 1: Random initialization ; 2: When the expected loss or number of training times is not completed, execute: 1): Get each batch of training samples ; 2): Calculate the results of the three branches The generating formulas are , and ; 3): Utilize Through the point set conversion formula generate , level set transformation formula generate ; 4) Prediction results By formula Generate point set representation features ; 5) Prediction results By formula Generate level set representation features ; 6) Prediction results By formula Generate pixel representation features ; 7) Prediction results By formula Generate point set representation features ; 8) Get the overall loss Through formulas (17), (21) and (22); 9) Update model parameters through back propagation ; 3: End the loop; 4: Return parameter ; For unlabeled data, although no label information can be used, the feature differences between different tasks can be minimized and task consistency between tasks can be found, thereby effectively utilizing unlabeled data.

[0058] Specifically, the loss function of the PS branch can be obtained in the same way as formula (10): : (19) Loss function of LS branch It can be expressed as: (20) The weight and Same as the supervised approach (as shown in formula (15)). Similarly, the strong and weak task consistency in UniTask+ (as shown in formula (15)) Figure 4 As shown, given by formula (16) for unlabeled data.

[0059] Therefore, the total loss of unlabeled data is It is expressed as follows: (twenty one) in, , , is the weight coefficient, preferably =0.06, =1.0, =0.01.

[0060] Finally, the entire loss of semi-supervised learning is expressed as: (twenty two) in, , is the weight coefficient, preferably =1.0.

[0061] In order to verify the effectiveness of the method of the present invention in medical image analysis, a publicly available human blastocyst dataset was selected to evaluate the performance of the UniTask+ framework of the present invention. At the same time, the versatility of the model was tested on the left atrium (LA) segmentation dataset provided by Zhaohan et al. Among them, the human blastocyst dataset contains 249 microscopic images of human blastocysts and their corresponding true (GT) labels, including the annotations of the main tissues of the zona pellucida (ZP), trophoblast (TE), inner cell mass (ICM) and blastocyst cavity provided by the Pacific Reproductive Medical Center. For the binary segmentation task, the main goal is to identify the inner cell mass because the ICM is the basis of embryonic development. For the multi-category segmentation task, the targets include ZP, TE, ICM and blastocyst cavity, and the size of the input image is adjusted to 256×256 in the test. The LA dataset contains 100 sets of gadolinium-enhanced magnetic resonance imaging (GE-MRIs) images and labels. It is the most commonly used dataset for evaluating semi-supervised performance. The size of the input image is adjusted to 112×112×80 in the test.

[0062] All tests were performed on PyTorch 1.10.0 using NVIDIA Tesla V100 GPU. To avoid overfitting, data augmentation methods were used to increase the diversity of training samples, including random scaling, random rotation, brightness and contrast enhancement. The maximum training epoch was set to 2000, the initial learning rate was set to 0.0001, and the weight decay was set to 0.00005. The Adam optimizer was selected in the experiment. To evaluate the performance of the network, the four evaluation criteria of ICM are usually used: accuracy, recall, Dice coefficient (Dice), and Jaccard index (Jaccard). For ICM segmentation, the original UNet was selected as the backbone network. For LA segmentation, the same four evaluation indicators were used: Dice, Jaccard index, average surface distance (ASD), and 95% Hausdorff distance (95HD). For experiments on dataset LA, VNet was selected as the backbone network.

[0063] In the test verification conducted on the human blastocyst dataset, 10% (19 scans) of the 190 training images were used as labeled data, and the remaining 171 images were used as unlabeled data. The UniTask+ framework method of the present invention achieved 97.67%, 92.76%, 92.09%, and 86.75% accuracy, recall, Dice, and Jaccard indicators, respectively. Compared with other existing methods, the UniTask+ framework method of the present invention improved Dice and Jaccard by at least 0.92% and 1.32%, respectively. When 50% of the labeled data was used for training, the UniTask+ framework method of the present invention achieved better results, reaching 98.37%, 95.28%, 94.76%, and 90.68% accuracy, recall, Dice, and Jaccard indicators, respectively; it was at least 1.42% and 1.85% better than other methods in Dice and Jaccard indicators.

[0064] In the left atrium (LA) segmentation dataset test experiment, 10% (8 scans) of the 80 training scans were used as labeled data, and the remaining 72 scans were used as unlabeled data. The UniTask+ framework method of the present invention achieved 90.61%, 83.03%, 6.80%, and 2.25% in Dice, Jaccard, ASD (voxel), and 95HD (voxel) score indicators, respectively, and its results were at least 1.78% and 2.72% better than other comparison methods in Dice and Jaccard indicators. When using 20% ​​of the labeled data for training, the UniTask+ framework method of the present invention achieved 92.74%, 86.55%, 4.87%, and 1.58% in Dice, Jaccard, ASD (voxel), and 95HD (voxel) scores, respectively, and its results were at least 1.43% and 2.47% better than other methods in Dice and Jaccard indicators, respectively.

[0065] The above test experiments show that there is inherent perturbation due to the need to abstract away fine pixel-level information during the segmentation process. Through these dual-task certification efforts, the UniTask+ framework approach promotes the exploration of intrinsic segmentation perturbations at three key levels: pixel level, geometric level, and point level. This enhances learning dynamics, benefiting both supervised and semi-supervised scenarios. The experimental evaluation of the present invention on ICM, blastocyst, and left atrium datasets has reached the state-of-the-art level. These outstanding performances demonstrate the effectiveness and robustness of the UniTask+ framework in addressing the complex challenges of medical image segmentation.

[0066] Through the detailed description of the specific implementation methods and embodiments of the present invention above, ordinary technicians in the field can understand that various changes, modifications, substitutions and variations can be made to these implementation methods and embodiments without departing from the principles and basic concepts of the present invention, and the scope of the present invention is defined by the attached claims and their equivalents. Contents not described in detail in this specification, such as the details of the specific structure and implementation of a typical neural network and its functional layer, are within the scope of what can be achieved by professional and technical personnel in the field based on the known prior art and the technical teachings of the present invention.

Claims

1. A biological tissue recognition and processing method based on task consistency, characterized in that Includes steps: A network model based on a unified task-aware consistency interaction framework is constructed, the network model comprising a skeleton network for extracting multi-scale image features, a main branch, and at least two additional branches; wherein the skeleton network is based on an encoder-decoder architecture of a deep learning model, and the output result of the skeleton network is input to the main branch and the at least two additional branches; the main branch and the at least two additional branches are a neural network in a parallel relationship, wherein the main branch is used to perform a main task, the task performed by at least one additional branch has a strong correlation with the main task, and the task performed by at least another additional branch has a weak correlation with the main task; Acquire training sample data, where the training sample data is biological tissue image data, and a portion of the biological tissue image data is labeled data for supervised training; Using the training sample data to train the network model based on the unified task perception consistency interaction framework, to obtain a biological tissue recognition processing training model; Use the biological tissue recognition and processing training model to recognize and process biological tissue image data.

2. The biological tissue identification and processing method based on task consistency as claimed in claim 1, characterized in that: The main branch is a segmentation branch, which is used to generate a pixel-level probability map and generate a pixel-level output result; An additional branch is the level set branch, which is used to generate the target level set representation and generate geometric output results, which are strongly related to the main task; Another additional branch is the point set branch, which is used to synthesize point representations related to the target structure and generate point-level output results that are weakly correlated with the main task.

3. The biological tissue identification and processing method based on task consistency as claimed in claim 2, characterized in that: Another part of the biological tissue image data is unlabeled data for semi-supervised training; the segmentation branch performs supervised training, and the level set branch and the point set branch perform supervised training and semi-supervised training simultaneously.

4. The biological tissue identification and processing method based on task consistency as claimed in claim 3, characterized in that: The biological tissue image data has dimensional structure, wherein C represents the number of color channels of the biological tissue image, H and W represent the height and width of the biological tissue image respectively, and R represents the real number space; the biological tissue image data is processed by the skeleton network to obtain feature sets of different scales Indicates that, the dimension , and Corresponding to the use of The downsampling factor of C, H, and W is processed; the integer D indicates the degree of downsampling performed at each stage.

5. The biological tissue identification and processing method based on task consistency as claimed in claim 4, characterized in that: The network model based on the unified task-aware consistency interaction framework adopts the loss function when training with labeled data for: ； in, is the loss function for supervised training of the segmentation branch, is the loss function for supervised training of the point set branch, is the loss function for supervised training of the level set branch, is the strong and weak task-aware consistency function, , , , is the weight coefficient.

6. The biological tissue identification and processing method based on task consistency as claimed in claim 5, characterized in that: The network model based on the unified task-aware consistency interaction framework adopts the loss function when training with unlabeled data for: ； in, is the loss function for unsupervised training of the point set branch, is the loss function for unsupervised training of the level set branch, is the strong and weak task-aware consistency function, , , is the weight coefficient.

7. The biological tissue identification and processing method based on task consistency according to claim 6, characterized in that: The loss function for semi-supervised learning of the network model based on the unified task-aware consistency interaction framework The overall expression is: ； in, , is the weight coefficient, The loss function used for training with labeled data is used for the network model based on the unified task-aware consistency interaction framework. The loss function used when training with unlabeled data is adopted for network models based on a unified task-aware consistency interaction framework.

8. The biological tissue identification and processing method based on task consistency as claimed in claim 7, characterized in that: Using the training sample data to train the network model based on the unified task-aware consistency interaction framework includes: Step S100: Randomly initialize segmentation branch parameters , point set branch parameter , level set branch parameter ; Step S200: When the predetermined loss expectation or training times are not completed, execute: Step S210: Obtain each batch of training samples ;in, , They are labeled datasets The corresponding input image and label map; For unlabeled dataset The corresponding input image; Step S220: Calculate the prediction results of the segmentation branch, point set branch, and level set branch respectively ; Step S230: Utilize Transformation formula through point set branching generate , through the transformation formula of the level set branch generate ; Step S240: Prediction results By formula Generate point set representation features ; Step S250: Prediction results By formula Generate level set representation features ; Step S260: Prediction results By formula Generate pixel representation features ; Step S270: Prediction results By formula Generate point set representation features ; Step S280: Obtaining the overall loss ; Step S290: Update model parameters by back propagation ; Step S300: When the predetermined loss expectation or training times are completed, the loop ends; Step S400: Return the obtained segmentation branch parameters , point set branch parameter , level set branch parameter .

Images