Image processing method, device and equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses an image processing method and device, equipment and a computer readable storage medium. The method comprises the following steps: acquiring label information corresponding to feature information of each voxel in a source domain image, and acquiring label information corresponding to feature information of each voxel in a target domain image; wherein the label information is used for representing the probability that the voxel belongs to a plurality of entity types; calculating difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image; updating feature information of each voxel in the target domain image based on the difference information to obtain updated feature information; and performing image recognition on the target domain image based on the updated feature information. Through the method, the accuracy of image recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to an image processing method, apparatus, device and computer-readable storage medium. Background Art

[0002] In the process of image processing, a specific machine learning model is needed for image recognition. The machine learning model is trained using source domain image data (which can be understood as sample data), so that the trained model can be used to recognize the image data in the target domain.

[0003] However, there are usually large differences between the source domain and the target domain (which can be understood as the image to be recognized), such as the feature differences between the source domain image and the target domain image caused by different scenes and acquisition devices. Such differences will lead to low model recognition ability and low image recognition accuracy.

[0004] Therefore, how to improve the accuracy of image recognition is an urgent problem to be solved. Summary of the invention

[0005] To solve the above technical problems, the embodiments of the present application provide an image processing method, apparatus, device and computer-readable storage medium, which can improve the accuracy of image recognition.

[0006] Among them, the technical solution adopted in this application is:

[0007] An image processing method, comprising:

[0008] Obtaining label information corresponding to feature information of each voxel in the source domain image, and obtaining label information corresponding to feature information of each voxel in the target domain image; wherein the label information is used to characterize the probability that the voxel belongs to multiple entity types;

[0009] Calculating difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image;

[0010] Based on the difference information, the feature information of each voxel in the target domain image is updated to obtain updated feature information;

[0011] Image recognition is performed on the target domain image based on the updated feature information.

[0012] An image processing device, comprising:

[0013] A computing unit, used to obtain label information corresponding to feature information of each voxel in a source domain image, and obtain label information corresponding to feature information of each voxel in a target domain image; wherein the label information is used to characterize the probability that the voxel belongs to multiple entity types;

[0014] The calculation unit is further configured to calculate difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image;

[0015] A processing unit, configured to update feature information of each voxel in the target domain image based on the difference information to obtain updated feature information;

[0016] A recognition unit is used to perform image recognition on the target domain image based on the updated feature information.

[0017] In one embodiment of the present application, based on the aforementioned scheme, the processing unit is further used to distinguish multiple voxels in the domain image to be determined according to foreground voxels and background voxels to obtain a foreground voxel set and a background voxel set; wherein the domain image to be determined is any one of the source domain image and the target domain image; the foreground voxel set and the background voxel set are respectively divided into multiple voxel sets; the feature set corresponding to each voxel set is calculated, and the feature set contains feature information of each voxel in the voxel set; the calculation unit is also used to calculate the label information corresponding to each voxel in the domain image to be determined based on the feature set of each voxel.

[0018] In one embodiment of the present application, based on the aforementioned scheme, an acquisition unit is used to obtain an initial label set corresponding to the voxel set to which each voxel belongs; the initial label set includes initial label information of each voxel, and the initial label information is used to characterize the predicted probability that the voxel belongs to the multiple entity types; the calculation unit is also used to calculate the label information corresponding to each voxel in the voxel set based on the initial label set, feature information of the multiple entity types and the feature set corresponding to the voxel set.

[0019] In one embodiment of the present application, based on the aforementioned scheme, the processing unit is further used to detect the image type to which the domain image to be determined belongs; based on multiple feature information corresponding to the image type, divide the multiple entity types; wherein each entity type corresponds to different feature information; based on the number of voxels contained in the voxel set, generate an initial label set corresponding to each voxel set.

[0020] In one embodiment of the present application, based on the aforementioned scheme, the computing unit is further used to sequentially multiply the initial label set, the feature information of the multiple entity types, and the feature set to obtain a first intermediate matrix; perform a weighted sum operation on the trace of the first intermediate matrix and the entropy function corresponding to the initial label set to obtain a second intermediate matrix; repeatedly multiply the second intermediate matrix, the feature information of the multiple entity types, and the feature set in sequence until the obtained second intermediate matrix satisfies a preset label condition; and use the second intermediate matrix that satisfies the preset label condition as the label set corresponding to the voxel set; wherein the label set contains the label information corresponding to each voxel.

[0021] In one embodiment of the present application, based on the aforementioned scheme, the processing unit is further used to determine the entity type to which the voxel belongs based on the label information of each voxel in the domain image to be determined; wherein the domain image to be determined is any one of the source domain image and the target domain image; and the calculation unit is further used to calculate the difference information between the source domain image and the target domain image based on the entity type to which each voxel belongs.

[0022] In one embodiment of the present application, based on the aforementioned scheme, the calculation unit is further used to calculate the probability distribution that the voxel belongs to the entity type based on the feature information of each voxel and the feature information of the multiple entity types, to obtain a first probability distribution; calculate the conditional probability distribution of the entity type in the source domain image data based on the feature information of each voxel in the source domain image data and the feature information of the multiple entity types, to obtain a second probability distribution; calculate the conditional probability distribution of the entity type in the target domain image data based on the feature information of each voxel in the target domain image data and the feature information of the multiple entity types, to obtain a third probability distribution; and calculate the difference information between the source domain image and the target domain image based on the first probability distribution, the second probability distribution and the third probability distribution.

[0023] In one embodiment of the present application, based on the aforementioned scheme, the computing unit is further used to calculate a first feature loss value based on the first probability distribution and the label set; calculate a second feature loss value based on the second probability distribution and the third probability distribution; and sum the first feature loss value and the second feature loss value to obtain difference information between the source domain image and the target domain image.

[0024] In one embodiment of the present application, based on the aforementioned scheme, the computing unit is further used to calculate a first identification loss value for the source domain image based on the second probability distribution; calculate a second identification loss value for the target domain image based on the third probability distribution; and perform a weighted sum operation on the first identification loss value and the second identification loss value to obtain the second feature loss value.

[0025] In one embodiment of the present application, based on the aforementioned scheme, the acquisition unit is further used to obtain a first feature expected value corresponding to the source domain image and a second feature expected value corresponding to the target domain image; the calculation unit is further used to sum a result of multiplying the first feature expected value by the first identification loss value and a result of multiplying the second feature expected value by the second identification loss value to obtain the second feature loss value.

[0026] In one embodiment of the present application, based on the above-mentioned solution, the processing unit is further used to use the entity type corresponding to the maximum probability value among multiple probability values ​​represented by the label information of the voxel as the entity type to which the voxel belongs.

[0027] An image processing device includes a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the above image processing method is implemented.

[0028] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor of a computer, enable the computer to execute the above image processing method.

[0029] A computer program product includes computer-readable instructions, and when the computer-readable instructions are executed by a processor, the above image processing method is implemented.

[0030] In the above technical solution:

[0031] The label information of the source domain image and the label information of the target domain image can characterize the probability that each voxel belongs to multiple entity types, so that the difference information between the source domain image and the target domain image can be accurately calculated. Based on the difference information, the feature information of each voxel in the target domain image can be updated to reduce or eliminate the difference between the target domain image and the source domain image. When the target domain image is subsequently recognized, it can be recognized by the updated feature information, so that the generalization ability of image recognition is improved, the recognition ability is guaranteed, and the accuracy of image recognition is improved.

[0032] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0034] Figure 1 It is a schematic diagram of an implementation environment involved in this application;

[0035] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment;

[0036] Figure 3 is a flowchart of an image processing method according to another exemplary embodiment;

[0037] Figure 4 is a flowchart of an image processing method according to another exemplary embodiment;

[0038] Figure 5 is a flowchart of an image processing method according to another exemplary embodiment;

[0039] Figure 6 is a flowchart of an image processing method according to another exemplary embodiment;

[0040] Figure 7 is a schematic diagram of label information of a voxel according to an exemplary embodiment;

[0041] Figure 8 is a schematic diagram of different domain images and their segmentation according to an exemplary embodiment;

[0042] Fig. 9 is a block diagram of an image processing device according to an exemplary embodiment;

[0043] Fig.10 The figure is a schematic diagram of the hardware structure of an image processing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0044] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments that are the same as the present application. Instead, they are only examples of devices and methods that are the same as some aspects of the present application as detailed in the attached claims.

[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, while some operations / steps can be integrated or partially integrated, so the actual execution order may change according to actual conditions.

[0047] It should be noted that the "multiple" mentioned in this application refers to two or more than two. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.

[0048] It should be noted that in the specific implementation of this application, user-related data is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0049] Before introducing the technical solutions of the embodiments of the present application, the technical terms involved in the embodiments of the present application are first introduced here.

[0050] Voxel is a volume element in three-dimensional space, similar to pixel in two-dimensional space. It is a cubic unit in three-dimensional space, used to represent discrete points in three-dimensional images or volume data. Voxel can be regarded as a small cube in three-dimensional space, with specific position coordinates and attribute values. Voxels are widely used to represent and process three-dimensional data in fields such as medical imaging, computer graphics, and computer-aided design. Each voxel can contain different information, such as grayscale value, color, density, etc., depending on the application field and data type. By combining and analyzing voxels, the shape, structure, and attribute information of three-dimensional objects can be restored.

[0051] MRI (Magnetic Resonance Imaging) is a medical imaging technique that uses strong magnetic fields and harmless radio waves to generate detailed images of the inside of the human body. MRI obtains images by exciting and detecting atomic nuclei inside the human body. When the human body is placed in a strong magnetic field, the atomic nuclei respond to the magnetic field. By applying radio wave pulses, the atomic nuclei can be excited to produce specific signals. After the received signals are processed, high-quality three-dimensional images can be generated, showing the different tissue structures inside the human body. MRI is able to provide high-contrast and high-resolution images, which can be used to observe and evaluate organs, tissues and structures inside the human body, such as the brain, spine, joints, heart, etc. It has a wide range of applications in medical diagnosis, disease monitoring and research, and is very valuable for detecting diseases, understanding tissue status and guiding surgery. In MRI, an image is composed of many voxels, which are arranged in three dimensions (usually x, y and z axes) into a three-dimensional voxel array. Each voxel contains the signal intensity or pixel value at that location, which represents the tissue characteristics of that area.

[0052] UDA (Unsupervised Domain Adaptation) is a machine learning algorithm used to solve the problem of different data distributions between different domains. Its goal is to improve the learning performance of target domain data by leveraging the knowledge of source domain data. These methods do this by finding shared features between the source and target domains and reducing the distribution differences between the domains. Unsupervised adaptive methods usually include steps such as feature extraction, domain adaptation, and model training. It is widely used in machine learning and computer vision tasks such as object detection, image classification, semantic segmentation, etc.

[0053] APD-Net (Adaptive Part Discovery Network) is used for image segmentation tasks. This method aims to automatically discover partial regions in an image and use them for feature extraction and prediction in segmentation tasks. APD-Net can adaptively learn and select important partial regions based on the characteristics of the input image to improve the accuracy and robustness of the segmentation results. The core idea of ​​APD-Net is to learn the important parts of the image and distinguish them from other regions through the attention mechanism and adaptive modules in the network. The network usually includes a feature extraction network and an adaptive module. The feature extraction network is used to extract the feature representation of the input image, while the adaptive module is used to select and adjust the weights of the features according to the attention mechanism. Through the adaptive part discovery network, APD-Net can effectively identify important parts from the image and use them in the image segmentation task, thereby improving the quality and accuracy of the segmentation results. This method has potential application value in image segmentation tasks in the field of computer vision.

[0054] In the field of domain adaptation and unsupervised learning, source domain and target domain are two important concepts.

[0055] The source domain refers to a dataset with existing labels or known categories. In the source domain, each sample has corresponding labels or category information. These labels can be used to train models or build algorithms to make predictions or classifications by learning the relationship between the features of the data and the labels. The source domain is usually the starting point from which the model acquires knowledge and experience.

[0056] The target domain refers to an unlabeled or only slightly labeled dataset. Unlike the source domain, the sample data in the target domain lacks label or category information, so it cannot be directly trained using supervised learning methods. The target domain is the dataset to which the model or algorithm trained based on the source domain data is applied. The target domain may have a different data distribution or feature distribution from the source domain, so there will be some differences.

[0057] In machine learning, a discriminator is a model or algorithm used to classify or judge input data. It is usually used to determine which category the input data belongs to or to predict a certain attribute of the input data. The main task of the discriminator is to determine whether the input data belongs to real data or generated data. During the training process, the generator tries to generate data samples similar to real data, while the discriminator tries to distinguish between real data and generated data and give corresponding classification results. Through continuous adversarial learning, the generator and the discriminator compete with each other and gradually improve their performance, eventually enabling the generator to generate more realistic data samples. The discriminator can be various types of models, such as neural networks, decision trees, support vector machines, etc. Its specific design depends on the nature of the task and the characteristics of the data.

[0058] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0059] In the related art, during the image processing process, there may be large differences between the source domain image used for training the model and the target domain image to be recognized, such as differences in features between the source domain image and the target domain image caused by different scenes and acquisition devices. Such differences will lead to low model recognition ability and low image recognition accuracy.

[0060] Based on this, the embodiments of the present application respectively propose an image processing method, an image processing apparatus, an image processing device, a computer-readable storage medium, and a computer program product. In these embodiments, the label information of each voxel can be calculated to reflect the probability that each voxel belongs to multiple entity types, and then the difference between the source domain image and the target domain image can be calculated, the difference can be reduced and eliminated, and the accuracy of image recognition can be improved.

[0061] See also Figure 1 , Figure 1 It is a schematic diagram of an implementation environment involved in this application.

[0062] Figure 1The implementation environment shown specifically includes a feature extractor 110 and a domain identifier 120. The feature extractor 110 and the domain identifier 120 may belong to the same machine learning algorithm or may belong to different machine learning algorithms, which is not limited here.

[0063] The feature extractor 110 can obtain the source domain image 130 and the target domain image 140, and perform feature extraction on each image. The feature extraction process can be based on a convolutional neural network, feature engineering, or other feature extraction methods. After feature extraction of the source domain image 130, the corresponding source domain feature information 150 is obtained. After feature extraction of the target domain image 140, the corresponding target domain feature information 160 is obtained. The feature extractor 110 can convert low-level features such as texture, shape, and color in the image into higher-level feature information, which can better express information such as the target, background, and context in the image. This feature information can be used for subsequent tasks, such as domain adaptation, transfer learning, or domain identification. The feature information can be a feature vector, which can be regarded as a vector representation of the source domain image 130 and the target domain image 140 in an abstract feature space.

[0064] The source domain image 130 may be source domain data with a label, and may include multiple images, each of which carries a label. The label may be a predefined category, which is used to indicate the type of label to which the image belongs. For example, the label of an image may indicate that the image is a heart MRI image, a prostate MRI image, a brain MRI image, etc., which is not limited here. In the embodiment of the present application, the source domain image 130 may be represented as Among them, s is the abbreviation of "source", X s,i is the i-th image in the source domain image 130, Y s,i is the label corresponding to the i-th image, n s is the number of images included in the source domain image 130.

[0065] The target domain image 140 may be unlabeled target domain data, or may include multiple images. That is, none of the multiple images in the target domain image 140 have predefined labels. In the embodiment of the present application, the target domain image 140 may be represented as Among them, t is the abbreviation of "target", X t,i is the i-th image in the target domain image 140, n t is the number of images included in the target domain image 140 .

[0066] The domain identifier 120 is a model or algorithm for distinguishing source domain feature information 150 from target domain feature information 160. Its purpose is to help achieve domain adaptation or domain identification tasks by learning the differences between the source domain and the target domain. Specifically, before applying the domain identifier 120, it is necessary to perform subclass mining on the source domain feature information 150 and the target domain feature information 160 respectively. Subclass mining here refers to generating label information corresponding to each voxel based on the feature information of each voxel. In this way, the source domain label information 170 corresponding to the source domain feature information 150 and the target domain label information 180 corresponding to the target domain feature information 160 can be obtained.

[0067] By mining subclasses and generating label information, each voxel can be associated with its category or type. That is, the label information corresponding to each voxel can reflect the probability that the voxel belongs to each type. These label information will play a key role in the training of the domain discriminator 120, helping the model understand the difference between the source domain image 130 and the target domain image 140, and learn the feature representation that distinguishes the two domains.

[0068] It should be noted that based on Figure 1 In the implementation environment shown, the present application can be applied to various fields that require reducing or eliminating the differences between source domain data and target domain data, including but not limited to the following fields:

[0069] Computer vision: In computer vision, such as image classification, object detection, image segmentation and other tasks, the source domain and the target domain may have different image feature distributions, such as images taken in different scenes and by different cameras. By reducing the difference between the source domain and the target domain, the performance of the model in the target domain can be improved, and tasks such as cross-domain image recognition and object detection can be achieved.

[0070] Natural language processing: In natural language processing, such as text classification, named entity recognition, machine translation and other tasks, the source domain and the target domain may have different text feature distributions, such as text data in different fields and languages. By reducing the difference between the source domain and the target domain, the generalization ability of the model in the target domain can be improved, and tasks such as cross-domain text classification and cross-language translation can be achieved.

[0071] Speech recognition: In the field of speech recognition, the source domain and the target domain may have different speech feature distributions, such as speech data from different speakers and different environments. By reducing the difference between the source domain and the target domain, the robustness of the model in the target domain can be improved, and tasks such as cross-domain speech recognition and speaker recognition can be achieved.

[0072] Medical image analysis: In the field of medical image analysis, the source domain and the target domain may contain image data from different medical institutions, different equipment, or different diseases. By reducing the difference between the source domain and the target domain, cross-domain learning and migration of medical images can be achieved, helping to improve the accuracy of disease diagnosis and the automatic analysis capabilities of medical images.

[0073] The present application embodiment is described using the field of medical image analysis as the target field, and using the prostate image segmentation scenario as the target scenario. Prostate image segmentation can also be called prostate segmentation, which refers to segmenting the prostate image and accurately extracting the prostate area in the image from the background or other tissue areas.

[0074] In clinical practice, automated segmentation of medical images (such as MRI) is of great significance for improving work efficiency and reducing manpower burden. Traditional rule-based or feature engineering methods have certain limitations, so deep learning methods such as convolutional neural networks (CNNs) have been widely used in medical image segmentation in recent years.

[0075] For prostate image segmentation, the goal of automatic segmentation can be achieved by training a convolutional neural network model. However, due to the differences in scanner brands, scanning parameters, image quality, and subject cohorts between different medical institutions, the distribution of image data may change. This distribution change may have an adverse effect on the trained model, causing its performance to deteriorate or even collapse on new datasets.

[0076] Therefore, in order to improve the robustness and generalization ability of the model, domain adaptation or domain identification is needed for different data distributions to reduce the difference between the source domain and the target domain. Such methods can help the model better adapt to new data distributions and achieve better performance in prostate image segmentation tasks in different medical institutions.

[0077] This application proposes a new adaptive part discovery network (APD-Net) for performing unsupervised domain adaptation (UDA) tasks to achieve prostate segmentation. Specifically, multiple cluster centers are introduced to replace traditional binary classifiers, and these cluster centers are used to segment image types into multiple entity types. In addition, in order to assist in the learning of cluster centers, a hybrid classification loss is designed. Unlike traditional marginal distribution matching, this application can match the distribution of multiple features with label information to effectively alleviate the problems caused by mismatching.

[0078] See also Figure 2 , Figure 2 is a flowchart of an image processing method according to an exemplary embodiment. The method can be applied to Figure 1The implementation environment shown in the figure is specifically executed by a computer. Of course, the method can also be applied to other implementation environments, and the execution subject of the method is not limited here.

[0079] The image processing method will be described in detail below using a computer as an exemplary execution subject.

[0080] like Figure 2 As shown, in an exemplary embodiment, the method comprises at least the following steps:

[0081] S210, obtaining label information corresponding to feature information of each voxel in the source domain image, and obtaining label information corresponding to feature information of each voxel in the target domain image; wherein the label information is used to represent the probability that the voxel belongs to multiple entity types.

[0082] In an embodiment of the present application, before obtaining label information, the type to which the source domain image and the target domain image belong can be subclassed and processed, and multiple entity types can be separated from the type. Each entity type can indicate an entity. For example, in the prostate segmentation scenario, for the source domain image and target domain image of the prostate type, the computer can continue to make a more fine-grained distinction for each image. Specifically, the computer can first distinguish each image according to the foreground and background, and then further divide the foreground area and the background area into multiple subclasses, each subclass representing a different entity type. In the prostate segmentation scenario, the multiple entity types in the divided foreground area can correspond to different areas or tissue structures of the prostate, such as the central zone, peripheral zone, prostate apex, etc. of the prostate, and the multiple entity types in the background area can correspond to organs or tissues around the prostate, such as bladder and muscle.

[0083] In the embodiment of the present application, the class c can be divided into M c entity types, we get M c A collection of entity types When c=0, it represents the foreground (such as the prostate), and when c=1, it represents the background (such as the bladder). In a class c, multiple feature information can be included. The computer can divide the foreground area into M 0 entity types, divided into M based on multiple feature information in the background area 1 entity type. 0 and M 1 They can be equal or unequal.

[0084] In the process of classifying entity types, the computer can select at least one feature information from multiple feature information contained in the foreground or background as the feature information that should correspond to an entity type. In other words, the computer can select one feature information from multiple feature information contained in the foreground or background as feature information of an entity type; or it can select more than one feature information, generate one feature information after fusion processing, and associate the generated feature information with a certain entity type. The specific selection method can be set by those skilled in the art, or the feature information of each entity type can also be designed by those skilled in the art, which is not limited here.

[0085] In the present embodiment, the K-means clustering method can be used to split the class c into multiple entity types.

[0086] Thus, M divided from class c c Each entity type has a corresponding feature information (which can be a feature vector). For multiple entity types of class c, they can be represented by a matrix, that is, Among them, each line Corresponding to the feature information of an entity type, D represents the dimension of the feature vector.

[0087] In this way, the computer can 0 feature information, which can be expressed as

[0088]

[0089] We can also get M in the background area 1 feature information, which can be expressed as

[0090]

[0091] Through the process of subclass mining, the computer can better understand the details and structures in the source domain images and the target domain images, and provide more accurate label information. This fine-grained distinction helps the model better learn the differences between the source domain images and the target domain images, thereby improving the accuracy and robustness of prostate segmentation.

[0092] It should be noted that the specific methods and strategies of subclass mining can be designed and adjusted according to the specific tasks and data characteristics. In prostate segmentation, the appropriate subclass division method can be determined based on medical knowledge and expert experience to ensure that the mined entity types can accurately represent the various parts and structures of the prostate and its surroundings.

[0093] In the embodiment of the present application, after receiving the source domain image and the target domain image, the computer can distinguish the multiple voxels in each image according to the foreground voxels and the background voxels to obtain the foreground voxel set and the background voxel set. That is, each image corresponds to its own foreground voxel set and background voxel set. Then, the foreground voxel set and the background voxel set can be divided into multiple voxel sets respectively, and the feature information of each voxel in each voxel set is calculated, so as to obtain the feature set corresponding to each voxel set.

[0094] Specifically, the computer can divide the voxels in the foreground and background areas into M parts, obtaining M voxel sets (which can be expressed as mini-batches). Each voxel set includes N c Each voxel corresponds to a feature information.

[0095] In this way, the computer can calculate the feature information corresponding to all voxels in an image, and these voxels are divided into multiple voxel sets, each voxel set corresponds to a feature set.

[0096] It can be understood that in the foreground area of ​​the source domain image, each voxel set contains voxels, in the background area, each voxel set contains In the foreground region of the target domain image, each voxel set contains voxels, in the background area, each voxel set contains Individual voxels.

[0097] In the embodiment of the present application, the computer may represent a feature set corresponding to a voxel set as: Among them, D represents the dimension of a feature information.

[0098] After obtaining the feature information corresponding to each voxel in a voxel set, an initial label information can be assigned to each voxel. The initial label information is used to characterize the predicted probability that each voxel belongs to multiple entity types, so that the initial label set corresponding to each voxel set can be obtained.

[0099] In the embodiment of the present application, since neither the source domain image nor the target domain image carries labels of each entity type, the labels for multiple entity types in the embodiment of the present application are actually pseudo-labels, and the computer can use the pseudo-labels as supervision to train the model for calculating the difference between the source domain image and the target domain image. Therefore, for training and learning, the computer can use pseudo-labels instead of real labels. Pseudo-labels are generated based on the prediction results of the model or other methods, and are used to represent the probability distribution of each voxel belonging to multiple entity types.

[0100] Optionally, the initial label information corresponding to each voxel can be a one-hot encoding of the voxel. That is, the entity type of the voxel can be randomly specified, and then the initial label information corresponding to the voxel is generated. For example, in a prostate segmentation scene, for a voxel in the foreground area, its entity type can be randomly determined to be a central band, and then the label of the voxel can be generated as {0, 0, ..., 1, 0, ..., 0, 0}, and only the entity type of the central band is 1.

[0101] In the embodiment of the present application, the initial label set corresponding to a voxel set can be expressed as in, is the one-hot encoding of the n-th voxel.

[0102] Furthermore, the computer can calculate the label information corresponding to each voxel in a voxel set based on the initial label set corresponding to a voxel set, the feature information of multiple entity types, and the feature set corresponding to the voxel set. By performing the same operation on each voxel set, the label information corresponding to the feature information of each voxel in the source domain image and the label information corresponding to the feature information of each voxel in the target domain image can be obtained.

[0103] S220, calculating difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image.

[0104] Specifically, the computer can determine the entity type to which each voxel belongs based on the label information corresponding to the source domain image and the label information corresponding to each voxel in the label information corresponding to the target domain image. The computer can discretize each label information and determine the mth voxel in a voxel set as the mth voxel. i entity types, where That is, the entity type corresponding to the maximum probability value among the multiple probability values ​​represented in the label information of the m-th voxel may be used as the entity type to which the voxel belongs.

[0105] After determining the entity type to which each voxel belongs, the computer can calculate the difference information between the source domain image data and the target domain image data based on the entity type to which each voxel belongs.

[0106] The computer can use the calculated pseudo-label (i.e. label information) as a supervisory signal to train the model. Specifically, during the training process, the model will make predictions based on the feature information of the source domain image and the target domain image, and compare the prediction results with the pseudo-label to calculate the difference between them, usually using the cross entropy loss function to measure the difference.

[0107] S230, updating feature information of each voxel in the target domain image based on the difference information to obtain updated feature information.

[0108] The computer can update the feature information of each voxel in the target domain image based on the calculated difference information, so that the feature information of the target domain image is more consistent with the source domain image, and obtain updated feature information. The updated feature information is feature information that reduces or eliminates the difference between the target domain image and the source domain image.

[0109] Optionally, the computer may update feature information of each voxel in the source domain image and the target domain image based on the calculated difference information.

[0110] S240, performing image recognition on the target domain image based on the updated feature information.

[0111] Since the updated feature information has reduced the difference between the source domain image and the target domain image, the model can more accurately analyze and judge the target domain image when performing image recognition. In this way, the model can better adapt to the feature distribution and structure of the target domain image and improve the accuracy and performance of image recognition.

[0112] Through this method, the difference between the source domain image and the target domain image can be reduced or eliminated, so that the model can better adapt to the data distribution of the target domain. This improves the generalization ability of the model, so that it has better performance on unseen target domain images. The model can more accurately identify objects, structures or specific features in the target domain image. The updated feature information enables the model to better understand and capture the key features in the target domain image, thereby improving the accuracy of image recognition.

[0113] In one embodiment of the present application, another image processing method is provided, which can be executed by a first device. Figure 3 As shown, the image processing method may include S310 to S340 and S220 to S240. That is, S310 to S340 are Figure 2 The specific implementation method of S210 is shown.

[0114] S310 to S340 are described below:

[0115] S310 , distinguishing a plurality of voxels in the to-be-determined domain image into foreground voxels and background voxels to obtain a foreground voxel set and a background voxel set.

[0116] The to-be-determined domain image is any one of the source domain image and the target domain image.

[0117] In this way, each image corresponds to its own set of foreground voxels and background voxels.

[0118] S320: Divide the foreground voxel set and the background voxel set into a plurality of voxel sets respectively.

[0119] Specifically, the computer can divide the voxels in the foreground and background areas into M parts, obtaining M voxel sets (which can be expressed as mini-batches). Each voxel set includes N c Each voxel corresponds to a feature information.

[0120] In this way, the computer can calculate the feature information corresponding to all voxels in an image, and these voxels are divided into multiple voxel sets, each voxel set corresponds to a feature set. This division method enables the computer to process multiple voxel sets at the same time.

[0121] It can be understood that in the foreground area of ​​the source domain image, each voxel set contains voxels, in the background area, each voxel set contains In the foreground region of the target domain image, each voxel set contains voxels, in the background area, each voxel set contains Individual voxels.

[0122] S330, calculating a feature set corresponding to each voxel set, where the feature set includes feature information of each voxel in the voxel set.

[0123] There are different ways to calculate the feature information of voxels, depending on the model used and the task requirements.

[0124] Alternatively, the computer can directly use the pixel value of the voxel as its feature information. The pixel value of each voxel can represent its grayscale, color or other physical properties in the image.

[0125] Optionally, the feature information of a voxel can be captured by considering its surrounding neighborhood pixels to capture local context information. Common methods include using a fixed-size sliding window or convolution kernel to calculate the statistical features of the pixels in the neighborhood (such as mean, variance, gradient, etc.).

[0126] Alternatively, the computer can use deep learning methods to learn feature representations of voxels through models such as convolutional neural networks. During training, the network automatically learns feature representations suitable for the task through a back-propagation algorithm.

[0127] The embodiment of the present application does not limit the method for calculating the feature information of voxels.

[0128] In the embodiment of the present application, the computer may represent a feature set corresponding to a voxel set as: Among them, D represents the dimension of a feature information.

[0129] S340, calculating label information corresponding to each voxel in the to-be-determined domain image based on the feature set of each voxel.

[0130] Specifically, S340 may include S341 to S342.

[0131] S341 to S342 are described below:

[0132] S341, obtaining an initial label set corresponding to the voxel set to which each voxel belongs.

[0133] The initial label set includes initial label information of each voxel, and the initial label information is used to represent the predicted probability that the voxel belongs to multiple entity types.

[0134] S342, calculating label information corresponding to each voxel in the voxel set based on the initial label set, feature information of multiple entity types, and a feature set corresponding to the voxel set.

[0135] The computer may calculate label information corresponding to each voxel in a voxel set based on an initial label set corresponding to a voxel set, feature information of multiple entity types, and a feature set corresponding to the voxel set.

[0136] Through this method, the computer can retain and utilize the diversity information between different voxels by dividing the source domain and target domain images into a foreground voxel set and a background voxel set, and further dividing them into multiple voxel sets. This can more comprehensively represent the features and structures in the image, which helps to improve the robustness and generalization ability of the model. For each voxel set, the computer can make the voxels in the voxel set have consistent label representation by calculating its corresponding label information. This helps to maintain the consistency of label distribution between the source domain and target domain images, reduce the interference caused by domain differences, and improve the performance of the model in the target domain. By calculating the feature information of the voxel set and calculating the label information based on the feature information, the alignment of the source domain and target domain image features can be achieved. This enables the model to better understand and capture the key features in the target domain image, thereby improving the accuracy of image recognition.

[0137] In one embodiment of the present application, another image processing method is provided, which can be executed by a computer. Figure 4 As shown, the image processing method is Figure 3 The specific implementation method of S341 shown may include S410 to S430.

[0138] S410 to S430 are described below:

[0139] S410, detecting the image type to which the to-be-determined domain image belongs.

[0140] The computer can determine the specific type of the image by analyzing the features of the image or other available information. For example, in the field of medical images, the image can be analyzed by analyzing the image's morphology, texture, color and other features to determine whether the image is of the prostate, heart or brain type.

[0141] S420: Divide into a plurality of entity types based on a plurality of feature information corresponding to the image type.

[0142] Among them, each entity type corresponds to different feature information.

[0143] The computer can classify and divide the entities in the domain image to be determined according to the characteristic information of each image type. Each entity type corresponds to different characteristic information, which is used to more accurately describe and represent the characteristics of each entity. For example, in the prostate segmentation scene, the multiple entity types in the divided foreground area can correspond to different areas or tissue structures of the prostate, such as the central zone, peripheral zone, prostate apex, etc. of the prostate, and the multiple entity types in the background area can correspond to organs or tissues around the prostate, such as bladder and muscle.

[0144] S430 , generating an initial label set corresponding to each voxel set based on the number of voxels contained in the voxel set.

[0145] A voxel set contains N c Therefore, for each voxel set, we can generate N c The initial label set of the initial label information of the voxels.

[0146] The initial label information corresponding to each voxel may be a one-hot encoding of the voxel, that is, the entity type of the voxel may be randomly assigned, and then the initial label information corresponding to the voxel may be generated.

[0147] In the embodiment of the present application, the initial label set corresponding to a voxel set can be expressed as in, is the one-hot encoding of the n-th voxel.

[0148] Through this method, the computer can generate an initial label information for each voxel, providing a supervisory signal for subsequent training and learning. The generation of initial label information can be based on random assignment or other methods, but does not need to rely on manual labeling. This can reduce the workload and time cost of manual labeling.

[0149] In one embodiment of the present application, another image processing method is provided, which can be executed by a computer. Figure 5 As shown, the image processing method is Figure 3 The specific implementation method of S342 shown may include S510 to S540.

[0150] S510 to S540 are described below:

[0151] S510: Multiply the initial label set, feature information of multiple entity types, and the feature set in sequence to obtain a first intermediate matrix.

[0152] Specifically, for a set of voxels in a foreground or background area, the computer may multiply an initial label set, feature information of multiple entity types, and a transpose of the feature set.

[0153] The first intermediate matrix can be calculated using the following formula:

[0154]

[0155] S520, performing a weighted sum operation on the trace of the first intermediate matrix and the entropy function corresponding to the initial label set to obtain a second intermediate matrix.

[0156] Specifically, the second intermediate matrix can be calculated using the following formula:

[0157]

[0158] The following constraints need to be met during the calculation:

[0159]

[0160] in, is the entropy function, ∈>0 is a preset hyperparameter. Tr() is the trace operator, which is to add all the values ​​on the diagonal of the first intermediate matrix. illustrate The value range of is positive real numbers.

[0161] S530, repeatedly multiplying the second intermediate matrix, the feature information of multiple entity types, and the feature set in sequence until the obtained second intermediate matrix meets the preset label condition.

[0162] The calculation process of the initial label set in the embodiment of the present application is a transportation problem, which can be solved using multiple Sinkhorn-Knopp iterations. The second intermediate matrix is ​​actually a matrix obtained after one iteration of the initial label set, so the second intermediate matrix can replace the initial label set for the next iteration.

[0163] In the embodiment of the present application, the number of iterations of the Sinkhorn-Knopp iterative method can be set by a person skilled in the art. When the number of iterations reaches a preset number of iterations, it can be considered that the second intermediate matrix meets the preset label condition.

[0164] S540: Use the second intermediate matrix that meets the preset label condition as a label set corresponding to the voxel set.

[0165] At this time, the second intermediate matrix that meets the preset label condition can accurately reflect the probability that each voxel belongs to each entity type, and thus can be used as the label set corresponding to the voxel set.

[0166] Through this method, the computer can use the Sinkhorn-Knopp iteration method to make the label information of each voxel gradually approach the actual label information from the initial label information as the number of iterations increases. By improving the accuracy of the label information of each voxel, the entity type to which each voxel belongs can be determined more accurately.

[0167] In one embodiment of the present application, another image processing method is provided, which can be executed by a first device. Figure 6 As shown, the image processing method may include S210, S610 to S620 and S230 to S240. That is, S610 to S620 are Figure 2 The specific implementation method of S220 is shown.

[0168] S610 to S620 are described below:

[0169] S610: Determine the entity type to which the voxel belongs based on the label information of each voxel in the domain image to be determined.

[0170] The to-be-determined domain image is any one of the source domain image and the target domain image.

[0171] Specifically, the entity type corresponding to the maximum probability value among the multiple probability values ​​represented by the label information of the voxel is taken as the entity type to which the voxel belongs.

[0172] For example, Figure 7 The following is a schematic diagram of the label information of a voxel. Figure 7In the above example, voxel i corresponds to the initial label information and label information (i.e., the label information obtained by multiple iterative calculations). In the initial label information, voxel i is randomly assigned to the entity type of the peripheral band. At this time, the initial label information is represented by {0, 1, 0, ..., 0, 0}. After multiple iterative calculations, the label information of voxel i is obtained, and the label information at this time is represented by {0.5, 0.1, 0.05, ..., 0.03, 0.01}. At this time, the probability value corresponding to the entity type of "central band" is the largest, so the entity type to which voxel i belongs can be determined as the central band.

[0173] S620: Calculate difference information between the source domain image and the target domain image based on the entity type to which each voxel belongs.

[0174] Specifically, S620 may include S621 to S624.

[0175] S621 to S624 are described below:

[0176] S621, based on the feature information of each voxel and the feature information of multiple entity types, calculate the probability distribution that the voxel belongs to the entity type to obtain a first probability distribution.

[0177] The probability distribution of a voxel belonging to an entity type is the probability that voxel i belongs to class M c The probability distribution of entity types can be expressed as:

[0178]

[0179] Among them, <,> is the calculation symbol of cosine similarity, and t is the temperature parameter. m |x i ) is the first probability distribution.

[0180] Optionally, in addition to using cosine similarity to calculate the first probability distribution, other metrics, such as Euclidean distance, etc., may also be used, which is not limited in the embodiments of the present application.

[0181] S622, calculating the conditional probability distribution of the entity type in the source domain image data based on the feature information of each voxel in the source domain image data and the feature information of the plurality of entity types, to obtain a second probability distribution.

[0182] The second probability distribution can be expressed as P(f s | m ).

[0183] S623, calculating the conditional probability distribution of the entity type in the target domain image data based on the feature information of each voxel in the target domain image data and the feature information of the plurality of entity types, to obtain a third probability distribution.

[0184] The third probability distribution can be expressed as P(f t |c m ).

[0185] S624: Calculate difference information between the source domain image and the target domain image based on the first probability distribution, the second probability distribution, and the third probability distribution.

[0186] Specifically, S624 may include S6241 to S6243.

[0187] S6241 to S6243 are described below:

[0188] S6241, calculate a first feature loss value based on the first probability distribution and the label set.

[0189] The computer can use a set of labels for a set of voxels As the first probability distribution P(c m |x i ) is supervised, and the cross entropy is used as the objective function to calculate the first feature loss value. The calculation formula of the first feature loss value can be based on P(c m |x i ) can be expressed as:

[0190] in yes The i-th row is the label information of the i-th voxel.

[0191] S6242: Calculate a second feature loss value based on the second probability distribution and the third probability distribution.

[0192] Specifically, the computer calculates the probability distribution P(f s |c m ) calculates the first discrimination loss value of the source domain image, which can be expressed as log(D([f s ,c m ])). And, the computer is based on the third probability distribution P(f t |c m ) calculates the second identification loss value for the target domain image, which can be expressed as log(1-D([f t ,c m ])).

[0193] Furthermore, the computer can perform a weighted sum operation on the first identification loss value and the second identification loss value to obtain a second feature loss value.

[0194] When calculating the value of the second feature, the computer also needs to obtain the expected value of the first feature corresponding to the source domain image (expressed as ), and the expected value of the second feature corresponding to the target domain image (expressed as ). The computer sums the result of multiplying the first feature expected value by the first identification loss value and the result of multiplying the second feature expected value by the second identification loss value to obtain the second feature loss value. can be expressed as log(D([f s ,c m ]))'s weight, can be expressed as log(1-D([f t ,c m ]))'s weight.

[0195] Finally, the calculation formula of the second feature loss value can be expressed as:

[0196]

[0197] S6243, summing the first feature loss value and the second feature loss value to obtain difference information between the source domain image and the target domain image.

[0198] That is, the calculation formula of the difference information is:

[0199] Difference information = L p +L d

[0200] Through this method, the computer can calculate the difference between the source domain image and the target domain image in a more fine-grained manner by considering the entity type to which each voxel belongs. This refined difference measurement can help capture subtle differences between entity types and improve the model's ability to distinguish different entity types. Based on the difference information calculated based on the entity type to which each voxel belongs, when the feature information of each voxel in the target domain image is subsequently updated, it can better adapt to the feature distribution of the target domain image, improve the performance of the model on the target domain image, and thus improve the image recognition accuracy of the model.

[0201] See also Figure 8 , Figure 8 FIG. 1 is an exemplary diagram of different domain images and their segmentation in an embodiment of the present application. Figure 8 As shown, mechanism 1 provides the original Figure 1 , using the corresponding mask recognition model, the mask 1 of the prostate can be obtained. Figure 2 , using the corresponding mask recognition model, the mask 2 of the prostate can be obtained.

[0202] It can be seen that the original images provided by institution 1 and institution 2 are quite different. Therefore, when using the mask recognition model for recognition, it is assumed that the model is based on the original images of institution 1. Figure 1 As the source domain image is trained, we now need to Figure 2 If it is used as the target domain image for recognition, the recognition results will be quite different, for example, Mask 1 and Mask 2 are quite different.

[0203] The computer can use the method provided in the embodiment of the present application to calculate the original Figure 1 Kahara Figure 2 The difference information between the two images can be used to reduce or eliminate the difference, thereby improving the accuracy of image recognition.

[0204] In the embodiment of the present application, Dice Similarity Coefficient (DSC) and 95% Hausdorff distance (95% HD) may be used as evaluation indicators, and the results are shown in Table 1.

[0205] method DSC↑ 95% HD↓ Baseline 51.26±0.92 59.33±2.72 This program 57.30±0.66 49.54±1.45

[0206] Table 1

[0207] DSC is a commonly used image similarity measurement method used to evaluate the similarity between two images or image segmentation results. It calculates the overlap between two sets. 95% HD is a method used to measure the distance between two images or image segmentation results. It is mainly used to evaluate the accuracy and precision of segmentation results.

[0208] The larger the DSC, the better, and the smaller the 95% HD, the better. As can be seen from Table 1, compared with the baseline, the technical solution of the present application has achieved gains in both evaluation indicators.

[0209] Fig. 9 FIG. 1 is a block diagram of an image processing device according to an embodiment of the present application. Fig. 9 As shown, the image processing device can be applied to a computer, and the device includes:

[0210] A calculation unit 910 is used to obtain label information corresponding to the feature information of each voxel in the source domain image, and obtain label information corresponding to the feature information of each voxel in the target domain image; wherein the label information is used to represent the probability that the voxel belongs to multiple entity types;

[0211] The calculating unit 910 is further configured to calculate difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image;

[0212] A processing unit 920 is used to update the feature information of each voxel in the target domain image based on the difference information to obtain updated feature information;

[0213] The recognition unit 930 is configured to perform image recognition on the target domain image based on the updated feature information.

[0214] In one embodiment of the present application, based on the aforementioned scheme, the processing unit 920 is further used to distinguish multiple voxels in the domain image to be determined according to foreground voxels and background voxels to obtain a foreground voxel set and a background voxel set; wherein the domain image to be determined is any one of the source domain image and the target domain image; the foreground voxel set and the background voxel set are respectively divided into multiple voxel sets; the feature set corresponding to each voxel set is calculated, and the feature set contains feature information of each voxel in the voxel set; the calculation unit 910 is also used to calculate the label information corresponding to each voxel in the domain image to be determined based on the feature set of each voxel.

[0215] In one embodiment of the present application, based on the aforementioned scheme, the acquisition unit 940 is used to obtain an initial label set corresponding to the voxel set to which each voxel belongs; the initial label set includes initial label information of each voxel, and the initial label information is used to characterize the predicted probability that the voxel belongs to multiple entity types; the calculation unit 910 is also used to calculate the label information corresponding to each voxel in the voxel set based on the initial label set, feature information of multiple entity types and the feature set corresponding to the voxel set.

[0216] In one embodiment of the present application, based on the aforementioned scheme, the processing unit 920 is also used to detect the image type to which the domain image to be determined belongs; based on multiple feature information corresponding to the image type, multiple entity types are divided; wherein each entity type corresponds to different feature information; based on the number of voxels contained in the voxel set, an initial label set corresponding to each voxel set is generated.

[0217] In one embodiment of the present application, based on the aforementioned scheme, the computing unit 910 is further used to sequentially multiply an initial label set, feature information of multiple entity types, and a feature set to obtain a first intermediate matrix; perform a weighted sum operation on the trace of the first intermediate matrix and the entropy function corresponding to the initial label set to obtain a second intermediate matrix; repeatedly multiply the second intermediate matrix, feature information of multiple entity types, and a feature set in sequence until the obtained second intermediate matrix satisfies a preset label condition; and use the second intermediate matrix that satisfies the preset label condition as a label set corresponding to a voxel set; wherein the label set contains label information corresponding to each voxel.

[0218] In one embodiment of the present application, based on the aforementioned scheme, the processing unit 920 is further used to determine the entity type to which the voxel belongs based on the label information of each voxel in the domain image to be determined; wherein the domain image to be determined is any one of the source domain image and the target domain image; the calculation unit 910 is further used to calculate the difference information between the source domain image and the target domain image based on the entity type to which each voxel belongs.

[0219] In one embodiment of the present application, based on the aforementioned scheme, the computing unit 910 is further used to calculate the probability distribution that the voxel belongs to the entity type based on the feature information of each voxel and the feature information of multiple entity types, to obtain a first probability distribution; calculate the conditional probability distribution of the entity type in the source domain image data based on the feature information of each voxel in the source domain image data and the feature information of multiple entity types, to obtain a second probability distribution; calculate the conditional probability distribution of the entity type in the target domain image data based on the feature information of each voxel in the target domain image data and the feature information of multiple entity types, to obtain a third probability distribution; and calculate the difference information between the source domain image and the target domain image based on the first probability distribution, the second probability distribution and the third probability distribution.

[0220] In one embodiment of the present application, based on the aforementioned scheme, the computing unit 910 is further used to calculate a first feature loss value based on the first probability distribution and the label set; calculate a second feature loss value based on the second probability distribution and the third probability distribution; and sum the first feature loss value and the second feature loss value to obtain difference information between the source domain image and the target domain image.

[0221] In one embodiment of the present application, based on the aforementioned scheme, the computing unit 910 is further used to calculate a first discrimination loss value for the source domain image based on a second probability distribution; calculate a second discrimination loss value for the target domain image based on a third probability distribution; and perform a weighted sum operation on the first discrimination loss value and the second discrimination loss value to obtain a second feature loss value.

[0222] In one embodiment of the present application, based on the aforementioned scheme, the acquisition unit 940 is further used to obtain a first feature expected value corresponding to the source domain image and a second feature expected value corresponding to the target domain image; the calculation unit 910 is further used to sum the result of multiplying the first feature expected value by the first identification loss value and the result of multiplying the second feature expected value by the second identification loss value to obtain a second feature loss value.

[0223] In one embodiment of the present application, based on the above solution, the processing unit 920 is further configured to use the entity type corresponding to the maximum probability value among multiple probability values ​​represented by the label information of the voxel as the entity type to which the voxel belongs.

[0224] It should be noted that the apparatus provided in the aforementioned embodiment and the method provided in the aforementioned embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment.

[0225] An embodiment of the present application also provides an image processing device, including: one or more processors; a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the above-mentioned image processing method.

[0226] Fig.10 It is a structural diagram of a computer system suitable for implementing the image processing device of the embodiment of the present application.

[0227] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0228] like Fig.10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003, such as executing the method in the above embodiment. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002 and the RAM 1003 are connected to each other through the bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0229] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0230] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1009, and / or installed from a removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0231] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable media may be, for example, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0232] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0233] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0234] Another aspect of the present application also provides a computer readable medium on which a computer program is stored, and when the computer program is executed by a processor, the image processing method as described above is implemented. The computer readable medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0235] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable medium. A processor of a computer device reads the computer instruction from the computer-readable medium, and the processor executes the computer instruction, so that the computer device executes the image processing method provided in each of the above embodiments.

[0236] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. An image processing method, characterized in that: include: Obtaining label information corresponding to feature information of each voxel in the source domain image, and obtaining label information corresponding to feature information of each voxel in the target domain image; wherein the label information is used to characterize the probability that the voxel belongs to multiple entity types; Calculating difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image; Based on the difference information, the feature information of each voxel in the target domain image is updated to obtain updated feature information; Image recognition is performed on the target domain image based on the updated feature information.

2. The method according to claim 1, characterized in that The step of obtaining label information corresponding to feature information of each voxel in the source domain image and obtaining label information corresponding to feature information of each voxel in the target domain image includes: Differentiate multiple voxels in the domain image to be determined according to foreground voxels and background voxels to obtain a foreground voxel set and a background voxel set; wherein the domain image to be determined is any one of the source domain image and the target domain image; Dividing the foreground voxel set and the background voxel set into a plurality of voxel sets respectively; Calculating a feature set corresponding to each voxel set, wherein the feature set includes feature information of each voxel in the voxel set; The label information corresponding to each voxel in the to-be-determined domain image is calculated based on the feature set of each voxel.

3. The method according to claim 2, characterized in that The calculating the label information corresponding to each voxel in the to-be-determined domain image based on the feature set of each voxel includes: Acquire an initial label set corresponding to the voxel set to which each voxel belongs; the initial label set includes initial label information of each voxel, and the initial label information is used to characterize the predicted probability that the voxel belongs to the multiple entity types; Based on the initial label set, the feature information of the multiple entity types and the feature set corresponding to the voxel set, label information corresponding to each voxel in the voxel set is calculated.

4. The method according to claim 3, characterized in that: The obtaining of an initial label set corresponding to a voxel set to which the voxel belongs includes: Detecting the image type to which the to-be-determined domain image belongs; Based on the plurality of feature information corresponding to the image type, the plurality of entity types are divided; wherein each entity type corresponds to different feature information; Based on the number of voxels contained in the voxel set, an initial label set corresponding to each voxel set is generated.

5. The method according to claim 4, characterized in that The calculating, based on the initial label set, the feature information of the plurality of entity types and the feature set corresponding to the voxel set, label information corresponding to each voxel in the voxel set comprises: Multiplying the initial label set, the feature information of the multiple entity types and the feature set in sequence to obtain a first intermediate matrix; Performing a weighted sum operation on the trace of the first intermediate matrix and the entropy function corresponding to the initial label set to obtain a second intermediate matrix; Repeating the multiplication of the second intermediate matrix, the feature information of the plurality of entity types, and the feature set in sequence until the obtained second intermediate matrix meets the preset label condition; The second intermediate matrix satisfying the preset label condition is used as the label set corresponding to the voxel set; wherein the label set contains label information corresponding to each voxel.

6. The method according to claim 1, characterized in that The calculating the difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image includes: Determining the entity type to which the voxel belongs based on the label information of each voxel in the to-be-determined domain image; wherein the to-be-determined domain image is any one of the source domain image and the target domain image; The difference information between the source domain image and the target domain image is calculated based on the entity type to which each voxel belongs.

7. The method according to claim 6, characterized in that The calculating and obtaining the difference information between the source domain image and the target domain image based on the entity type to which each voxel belongs respectively includes: Based on the feature information of each voxel and the feature information of the multiple entity types, calculating the probability distribution of the voxel belonging to the entity type to obtain a first probability distribution; Calculate the conditional probability distribution of the entity type in the source domain image data based on the feature information of each voxel in the source domain image data and the feature information of the multiple entity types to obtain a second probability distribution; Calculate the conditional probability distribution of the entity type in the target domain image data based on the feature information of each voxel in the target domain image data and the feature information of the multiple entity types to obtain a third probability distribution; Based on the first probability distribution, the second probability distribution and the third probability distribution, difference information between the source domain image and the target domain image is calculated.

8. The method according to claim 7, characterized in that The calculating, based on the first probability distribution, the second probability distribution, and the third probability distribution, difference information between the source domain image and the target domain image includes: Calculate a first feature loss value based on the first probability distribution and the label set; Calculate a second feature loss value based on the second probability distribution and the third probability distribution; The first feature loss value and the second feature loss value are summed to obtain difference information between the source domain image and the target domain image.

9. The method according to claim 8, characterized in that The calculating a second feature loss value based on the second probability distribution and the third probability distribution includes: Calculate a first discrimination loss value for the source domain image based on the second probability distribution; Calculating a second discrimination loss value for the target domain image based on the third probability distribution; A weighted sum operation is performed on the first identification loss value and the second identification loss value to obtain the second feature loss value.

10. The method according to claim 9, characterized in that The performing a weighted sum operation on the first identification loss value and the second identification loss value to obtain the second feature loss value includes: Acquire a first feature expected value corresponding to the source domain image and a second feature expected value corresponding to the target domain image; A result of multiplying the first feature expected value by the first identification loss value and a result of multiplying the second feature expected value by the second identification loss value are summed to obtain the second feature loss value.

11. The method according to claim 6, characterized in that The determining the entity type to which each voxel belongs based on the label information of each voxel in the to-be-determined domain image comprises: The entity type corresponding to the maximum probability value among the multiple probability values ​​represented by the label information of the voxel is used as the entity type to which the voxel belongs.

12. An image processing device, characterized in that: include: A computing unit, used to obtain label information corresponding to feature information of each voxel in a source domain image, and obtain label information corresponding to feature information of each voxel in a target domain image; wherein the label information is used to characterize the probability that the voxel belongs to multiple entity types; The calculation unit is further configured to calculate difference information between the source domain image and the target domain image based on the label information corresponding to the source domain image and the label information corresponding to the target domain image; A processing unit, configured to update feature information of each voxel in the target domain image based on the difference information to obtain updated feature information; A recognition unit is used to perform image recognition on the target domain image based on the updated feature information.

13. An image processing device, characterized in that: include: a memory storing computer-readable instructions; A processor reads the computer-readable instructions stored in the memory to execute the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program, the computer program is stored in a computer-readable storage medium, a processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device performs the method according to any one of claims 1 to 11.

Images