Image sample domain alignment method and device based on adversarial learning

Abstract

The invention discloses an image sample domain alignment method and device based on adversarial learning, and the method comprises the steps: obtaining a plurality of image sample domains, and extracting the target dimension data of each image sample domain; and taking the target dimension data as input data, inputting the input data into a pre-constructed sample domain confrontation module for data processing to obtain aligned sample domain data output by the sample domain confrontation module, and then updating and optimizing a feature extraction network through a back propagation gradient. The robustness of downstream tasks (such as image classification and the like) of a feature extraction network on different domain image feature recognition is improved. According to the method, adversarial training is carried out on a plurality of different sample domains, and image data alignment among different sample domains is realized, so that the problems of high acquisition difficulty and long acquisition period of a data set for deep learning model training caused by relatively large image difference among different samples and batches of medical image data are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an image sample domain alignment method and device based on adversarial learning. Background Art

[0002] Medical image processing, detection and recognition methods based on deep learning greatly reduce manual labor and improve recognition accuracy and efficiency. They have now developed into important tools in the field of medical image analysis. Accordingly, the acquisition and use of medical image data has become a key link in building effective deep learning algorithms. The acquisition and use of medical image data have their own unique dilemmas. In addition to the difficulties in data acquisition and sharing caused by issues such as the privacy of medical patients, in medical image data sets such as blood smears, there are often problems of single sample sources and high sample homogeneity, that is, the number of collected images is far greater than the number of samples used for image collection, which makes the data set show the characteristics of high similarity between images and poor overall representativeness, which in turn affects the application of upper-level deep learning methods and brings problems such as poor generalization of methods and large fluctuations in indicators such as accuracy of methods.

[0003] In view of this, a method and device for image sample domain alignment based on adversarial learning are provided to solve the problem of difficulty and long acquisition cycle in obtaining data sets for deep learning model training due to large differences between different samples and batches of medical image data. Summary of the invention

[0004] To this end, an embodiment of the present invention provides an image sample domain alignment method and device based on adversarial learning, which realizes image data alignment between different sample domains by performing adversarial training on multiple different sample domains, thereby solving the problem of difficulty and long acquisition cycle of data sets for deep learning model training due to large differences between different samples and batches of medical image data. The plug-and-play sample domain (sample source) alignment method based on adversarial learning can improve the sensitivity of deep learning models to specific sample distributions when processing images from different sample sources, and to a certain extent solve the limitation of single sample sources and high homogeneity on the application of deep learning methods in open real environments.

[0005] In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:

[0006] The present invention provides an image sample domain alignment method based on adversarial learning, the method comprising:

[0007] Acquire multiple image sample domains, and extract target dimension data of each image sample domain;

[0008] The target dimension data is input as input data into a pre-built sample domain adversarial module for data processing to obtain aligned sample domain data output by the sample domain adversarial module, and then the feature extraction network is optimized by back-propagation gradient update;

[0009] Among them, the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

[0010] In some embodiments, during the training process of the sample domain adversarial module:

[0011] The gradient reversal layer is used to change the gradient optimization direction of the input data;

[0012] The projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction.

[0013] In some embodiments, changing the gradient optimization direction of the input data by a gradient reversal layer specifically includes:

[0014] During the forward propagation of the sample domain adversarial module, the input tensor and output tensor of the gradient reversal layer remain consistent;

[0015] During the back propagation process of the sample domain adversarial module, when the model parameters are updated according to the gradient of the supervised contrast loss function after the gradient reversal layer, the gradients passing through the gradient reversal layer are all reversed.

[0016] In some embodiments, the projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction, specifically including:

[0017] Using the feature extraction model to obtain the high-dimensional feature vector of the input image;

[0018] The high-dimensional feature vector of the image is input into the projection network, and the high-dimensional feature vector of the image is subjected to dimensionality reduction processing according to the hidden layer weights of the perceptron, the weight parameters of the output layer and the hidden layer activation function to obtain an output vector.

[0019] In some embodiments, obtaining the output vector further comprises:

[0020] Regularization is performed on the output vector to obtain a regularized output vector.

[0021] In some embodiments, the supervised contrast loss L con The expression is:

[0022]

[0023] Where i represents the i-th image, and I is the set of all medical images of this type used for training;

[0024] A(i) is the set of all images except i, and P(i) is the set of all images except i that are consistent with the sample domain of i.

[0025] In some embodiments, the target dimension data includes image data, label data, and sample domain information data.

[0026] The present invention also provides an image sample domain alignment device based on adversarial learning, the device comprising:

[0027] A data acquisition unit, used to acquire multiple image sample domains and extract target dimension data of each image sample domain;

[0028] A data processing unit, configured to input the target dimension data as input data into a pre-built sample domain adversarial module for data processing, so as to obtain aligned sample domain data output by the sample domain adversarial module, and then optimize the feature extraction network by back-propagation gradient updating;

[0029] Among them, the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

[0030] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0032] The image sample domain alignment method and device based on adversarial learning provided by the present invention obtain multiple image sample domains and extract target dimension data of each image sample domain; the target dimension data is input as input data into a pre-constructed sample domain adversarial module for data processing to obtain aligned sample domain data output by the sample domain adversarial module, and then the feature extraction network is optimized by back-propagation gradient update to improve the robustness of downstream tasks (such as image classification and other tasks) of the feature extraction network for image feature recognition in different domains; wherein the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence. The method and device realize image data alignment between different sample domains by performing adversarial training on multiple different sample domains, thereby solving the problem of difficulty and long acquisition cycle of data sets used for deep learning model training due to large differences in images between different samples and batches of medical image data. The plug-and-play sample domain (sample source) alignment method based on adversarial learning can improve the sensitivity of the deep learning model to the specific sample distribution when processing images from different sample sources, and to a certain extent solve the limitations of the application of deep learning methods in open real-world environments due to the single and high homogeneity of sample sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0034] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0035] Figure 1 One of the flow charts of the image sample domain alignment method based on adversarial learning provided by the present invention;

[0036] Figure 2 The second flowchart of the image sample domain alignment method based on adversarial learning provided by the present invention;

[0037] Figure 3 Flow chart 3 of the image sample domain alignment method based on adversarial learning provided by the present invention;

[0038] Figure 4 A structural block diagram of the image sample domain alignment device based on adversarial learning provided by the present invention;

[0039] Figure 5 This is a structural block diagram of a computer device provided by the present invention. DETAILED DESCRIPTION

[0040] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] For medical image data, especially blood smear images that require staining and other operations, the images of different samples and batches vary greatly. It takes a lot of data to obtain a sufficiently representative data set, which is difficult to obtain and takes a long time to obtain. In order to solve this technical problem, the present invention provides an image sample domain alignment method and device based on adversarial learning. Figure 1-Figure 4 The image sample domain alignment method and device based on adversarial learning provided by the present invention are introduced.

[0042] Please refer to Figure 1 , Figure 1 This is one of the flowcharts of the image sample domain alignment method based on adversarial learning provided by the present invention.

[0043] In a specific implementation, the image sample domain alignment method based on adversarial learning provided by the present invention comprises the following steps:

[0044] S110: Acquire multiple image sample domains, and extract target dimension data of each image sample domain;

[0045] S120: inputting the target dimension data as input data into a pre-built sample domain adversarial module for data processing to obtain aligned sample domain data output by the sample domain adversarial module, and then optimizing the feature extraction network by back-propagation gradient update to improve the robustness of downstream tasks (such as image classification tasks) of the feature extraction network to feature recognition of images in different domains;

[0046] Among them, the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

[0047] It should be understood that the sample domain adversarial module provided by the present invention can be used in combination with typical convolution classification and detection models in deep learning, such as Figure 2 As shown in Figure 2, the sample domain adversarial module is a plug-and-play method. Except for some changes in the data loading process, the use of this method does not affect the original structure and training and reasoning process of most classification and detection models in the medical background. It is particularly suitable for any image feature information extraction based on convolutional neural networks (see Figure 2 Since the use of this method is independent of the classification and detection models in most cases, the description of how the model implements classification, detection and other tasks is omitted in this embodiment. Figure 2 The implementation details related to the sample domain adversarial module in the shown process are introduced in detail.

[0048] In some embodiments, the target dimension data includes image data, label data, and sample domain information data. After the image is appropriately transformed and enhanced, the image and its corresponding annotation information are input into the subsequent model for training or reasoning (no annotation information is required only for reasoning). In this embodiment, for medical images, common annotation information may include pathological typing (classification task), location distribution of diseased areas in the image (detection task), etc.; the sample domain adversarial module is trained based on the image sample domain (i.e., the biological sample source of the aforementioned medical image); therefore, Figure 2 As shown in Figure 2, in the data loading stage, in addition to the images and medical annotation data required for general classification and detection training, the sample domain information of the data must also be loaded, that is, the information record that identifies the biological sample source of the collected image. Figure 2 The sample domain number S represents it.

[0049] In some embodiments, during the training process of the sample domain adversarial module, the gradient reversal layer is used to change the gradient optimization direction of the input data, and the projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction.

[0050] like Figure 2In the illustrated embodiment, the processing flow of the reference input data of the sample domain adversarial module is composed of three parts from left to right: a gradient reversal layer (GRL), a projection network, and a supervised contrast loss calculation. Among them, the gradient reversal layer is a common operation in adversarial learning, which is used to reverse the gradient optimization direction and move the model parameters of each layer before GRL in the direction of increasing the loss function; the projection network and the supervised contrast loss are similar to the discriminator and the binary cross entropy loss in general feature adversarial learning, respectively, and are commonly used in representation learning, in which they are used to improve the discrimination of samples with different labels in the feature representation space.

[0051] In the application scenario of this embodiment, these two parts are combined with the GRL layer to achieve a confrontation effect, that is, the model feature extraction part ( Figure 2 The feature representation of medical images extracted from the backbone part of the model (the backbone part in the model) evolves towards a more general and more inseparable direction (in the sample domain), that is, the backbone part of the model is more inclined to extract image features in the image that are not related to the source of the biological sample.

[0052] Among them, changing the gradient optimization direction of the input data through the gradient reversal layer specifically includes:

[0053] During the forward propagation of the sample domain adversarial module, the input tensor and output tensor of the gradient reversal layer remain consistent;

[0054] During the back propagation process of the sample domain adversarial module, when the model parameters are updated according to the gradient of the supervised contrast loss function after the gradient reversal layer, the gradients passing through the gradient reversal layer are all reversed.

[0055] That is to say, the gradient reversal layer (GRL layer) is used to change the direction of gradient optimization, and its effect can be discussed separately in the forward and backward propagation processes. In the forward propagation process of the model, the GRL layer does not change the input tensor x passing through the layer, that is, the input and output tensors remain consistent:

[0056] GRL:x→x

[0057] In the back propagation process, according to the supervised contrast loss function L after this layer con (See Figure 1 ) When the gradient updates the model parameters, the gradient passing through this layer will be negated; that is, Figure 1 The model parameters θ of the right side of the layer and the projection network part prj Update along the direction that makes the loss function decrease, and let the model parameter θ of the left side of the layer and the feature extraction part f Update along the direction that makes the loss function rise. This process can be expressed as follows:

[0058]

[0059] Among them, t is the number of iterative optimizations, η is the learning rate parameter, and λ is the supervised contrast loss weight. The latter two are determined based on experience or hyperparameter search.

[0060] When the projection network is used to reduce the dimensionality of the high-dimensional feature vector of the image obtained by feature extraction, as shown in FIG. Figure 3 As shown, the specific steps include:

[0061] S310: Obtaining a high-dimensional feature vector of the input image using a feature extraction model;

[0062] S320: Input the high-dimensional feature vector of the image into the projection network, and perform dimensionality reduction processing on the high-dimensional feature vector of the image according to the hidden layer weight of the perceptron, the weight parameter of the output layer and the hidden layer activation function to obtain an output vector;

[0063] S330: Perform regularization processing on the output vector to obtain a regularized output vector.

[0064] Specifically, the projection network is used to compress and reduce the dimensionality of the high-dimensional feature vector of the image obtained in the feature extraction part, and project it into a low-dimensional feature space to avoid the failure of the subsequent contrast loss calculation based on the cosine distance. This part is relatively flexible. Here, a perceptron implementation with one hidden layer is described. The input is the vector x of the medical image feature map obtained by the feature extraction part after global average pooling, one-dimensionalization, and GRL layer processing. f , the output is another vector x of fixed dimension (usually 128 dimensions) prj , and in order to facilitate the calculation of supervised contrast loss, the output vector is generally regularized, represented by z below, to distinguish it from the vector without regularization; thus, the process can be formally expressed as:

[0065]

[0066] Among them, ‖·‖ represents the modulus of the vector, They are the hidden layer weights and output layer weight parameters of the perceptron (d f , d h and d o The vector x fDimension and number of neurons in the hidden layer and output layer of the perceptron), A ReLU ReLU is used as the hidden layer activation function.

[0067] Supervised contrast loss is commonly used in fields such as representation learning, with image category labels as supervisory information. In the problems addressed by this method, since the number of sample domains (i.e., the biological sample source of medical images) is large and may continue to grow with the expansion of the data set, and the number of images contained in each sample domain is generally quite different, that is, there is an obvious long-tail problem. Therefore, this method uses a projection network and a supervised contrast loss based on cosine distance as a representation learning method to replace the discriminator and classification loss used in general adversarial learning tasks. At the same time, since the purpose of this method is to learn a model that is highly generalizable and common to all sample domains, the sample domain information of the image is used as supervision in this method, and the supervised contrast loss L con The expression is:

[0068]

[0069] Where i represents the i-th image, I is the set of all medical images of this type used for training, A(i)≡I\{i}, is the set of all images except i, and P(i)≡{p∈A(i)∶S p =S i}, is the set of all images except i that are consistent with the sample domain of i (i.e., the source of biological samples is consistent). Minimize the supervised contrast loss L con It is required that the image feature representations in the same sample domain (from the same biological sample source) are as close as possible, while the image feature representations in different sample domains (from different biological sample sources) are far away from each other; therefore, along L con The projection network optimized in the descending direction will try its best to transform the image features x f According to the source of the sample domain, on the contrary, after passing through the GRL layer, along the L con The feature extraction part of the ascending direction optimization will minimize L sup Under the constraint of f The distance between them is closer and more difficult to distinguish.

[0070] In summary, the training process of the adversarial module of the present invention is as follows: first, the input image sample domain data will pass through the gradient inversion layer, which is used to adjust the gradient optimization direction of the input data. Then, the projection network will reduce the dimensionality of the high-dimensional feature vector of the image after feature extraction to generate a low-dimensional feature vector, which is the output of the sample domain adversarial module. Throughout the process, the network parameters are optimized through supervised contrast loss (Supervised Contrast ive Loss) to make the image features from the same sample domain as close as possible, while the image features from different sample domains are as far away as possible. Finally, through multiple iterations and parameter updates, the model can extract features that are insensitive to the sample domain, thereby achieving the goal of sample domain alignment. In the above specific implementation manner, the image sample domain alignment method based on adversarial learning provided by the present invention obtains multiple image sample domains and extracts target dimension data of each image sample domain; the target dimension data is input as input data into a pre-constructed sample domain adversarial module for data processing to obtain the aligned sample domain data output by the sample domain adversarial module, and then the feature extraction network is optimized by back-propagation gradient update to improve the robustness of downstream tasks of the feature extraction network (such as image classification and other tasks) for feature recognition of images in different domains; wherein the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

[0071] This method performs adversarial training on multiple different sample domains to achieve image data alignment between different sample domains, thereby solving the problem of difficulty and long acquisition cycle of data sets for deep learning model training due to large differences between different samples and batches of medical image data. The plug-and-play sample domain (sample source) alignment method based on adversarial learning can improve the sensitivity of deep learning models to specific sample distributions when processing images from different sample sources, and to a certain extent solve the limitations of single sample sources and high homogeneity on the application of deep learning methods in open real-world environments. In addition, by reducing the negative impact of the single biological sample source and high homogeneity of the data set on the training of deep learning models, the generalization and robustness of deep learning methods in medical image applications are improved.

[0072] In addition to the above method, the present invention also provides an image sample domain alignment device based on adversarial learning, such as Figure 4 As shown, the device comprises:

[0073] A data acquisition unit 410 is used to acquire multiple image sample domains and extract target dimension data of each image sample domain;

[0074] A data processing unit 420 is used to input the target dimension data as input data into a pre-built sample domain adversarial module for data processing, so as to obtain aligned sample domain data output by the sample domain adversarial module, and then optimize the feature extraction network by back-propagation gradient update;

[0075] Among them, the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

[0076] In some embodiments, during the training process of the sample domain adversarial module:

[0077] The gradient reversal layer is used to change the gradient optimization direction of the input data;

[0078] The projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction.

[0079] In some embodiments, changing the gradient optimization direction of the input data by a gradient reversal layer specifically includes:

[0080] During the forward propagation of the sample domain adversarial module, the input tensor and output tensor of the gradient reversal layer remain consistent;

[0081] During the back propagation process of the sample domain adversarial module, when the model parameters are updated according to the gradient of the supervised contrast loss function after the gradient reversal layer, the gradients passing through the gradient reversal layer are all reversed.

[0082] In some embodiments, the projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction, specifically including:

[0083] Using the feature extraction model to obtain the high-dimensional feature vector of the input image;

[0084] The high-dimensional feature vector of the image is input into the projection network, and the high-dimensional feature vector of the image is subjected to dimensionality reduction processing according to the hidden layer weights of the perceptron, the weight parameters of the output layer and the hidden layer activation function to obtain an output vector.

[0085] In some embodiments, obtaining the output vector further comprises:

[0086] Regularization is performed on the output vector to obtain a regularized output vector.

[0087] In some embodiments, the supervised contrast loss L con The expression is:

[0088]

[0089] Where i represents the i-th image, and I is the set of all medical images of this type used for training;

[0090] A(i) is the set of all images except i, and P(i) is the set of all images except i that are consistent with the sample domain of i.

[0091] In some embodiments, the target dimension data includes image data, label data, and sample domain information data.

[0092] In the above specific implementation manner, the image sample domain alignment device based on adversarial learning provided by the present invention obtains multiple image sample domains and extracts target dimension data of each image sample domain; the target dimension data is input as input data into a pre-constructed sample domain adversarial module for data processing to obtain the aligned sample domain data output by the sample domain adversarial module, and then the feature extraction network is optimized by back-propagation gradient update to improve the robustness of downstream tasks of the feature extraction network (such as image classification and other tasks) for feature recognition of images in different domains; wherein the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

[0093] The device performs adversarial training on multiple different sample domains to achieve image data alignment between different sample domains, thereby solving the problem of difficulty and long acquisition cycle of data sets for deep learning model training due to large differences between different samples and batches of medical image data. The plug-and-play sample domain (sample source) alignment method based on adversarial learning can improve the sensitivity of deep learning models to specific sample distributions when processing images from different sample sources, and to a certain extent solve the limitation of single sample sources and high homogeneity on the application of deep learning methods in open real-world environments.

[0094] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiment are implemented.

[0095] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0096] Corresponding to the above embodiment, the embodiment of the present invention further provides a computer storage medium, which contains one or more program instructions, wherein the one or more program instructions are used to execute the above method.

[0097] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the above method.

[0098] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0099] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0100] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0101] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0102] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0103] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0104] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0105] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for image sample domain alignment based on adversarial learning, characterized in that: The method comprises: Acquire multiple image sample domains, and extract target dimension data of each image sample domain; The target dimension data is input as input data into a pre-built sample domain adversarial module for data processing to obtain aligned sample domain data output by the sample domain adversarial module, and then the feature extraction network is optimized by back-propagation gradient update; Among them, the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

2. The image sample domain alignment method based on adversarial learning according to claim 1, characterized in that: During the training process of the sample domain adversarial module: The gradient reversal layer is used to change the gradient optimization direction of the input data; The projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction.

3. The image sample domain alignment method based on adversarial learning according to claim 2, characterized in that: Changing the gradient optimization direction of the input data through the gradient reversal layer specifically includes: During the forward propagation of the sample domain adversarial module, the input tensor and output tensor of the gradient reversal layer remain consistent; During the back propagation process of the sample domain adversarial module, when the model parameters are updated according to the gradient of the supervised contrast loss function after the gradient reversal layer, the gradients passing through the gradient reversal layer are all reversed.

4. The image sample domain alignment method based on adversarial learning according to claim 2, characterized in that: The projection network is used to perform dimensionality reduction processing on the high-dimensional feature vector of the image obtained by feature extraction, specifically including: Using the feature extraction model to obtain the high-dimensional feature vector of the input image; The high-dimensional feature vector of the image is input into the projection network, and the high-dimensional feature vector of the image is subjected to dimensionality reduction processing according to the hidden layer weights of the perceptron, the weight parameters of the output layer and the hidden layer activation function to obtain an output vector.

5. The image sample domain alignment method based on adversarial learning according to claim 4, characterized in that: The output vector is obtained, and then: Regularization is performed on the output vector to obtain a regularized output vector.

6. The image sample domain alignment method based on adversarial learning according to claim 1, characterized in that: The supervised contrast loss L con The expression is: Where i represents the i-th image, and I is the set of all medical images of this type used for training; A(i) is the set of all images except i, and P(i) is the set of all images except i that are consistent with the sample domain of i.

7. The image sample domain alignment method based on adversarial learning according to claim 1, characterized in that: The target dimension data includes image data, label data and sample domain information data.

8. An image sample domain alignment device based on adversarial learning, characterized in that: The device comprises: A data acquisition unit, used to acquire multiple image sample domains and extract target dimension data of each image sample domain; A data processing unit, configured to input the target dimension data as input data into a pre-built sample domain adversarial module for data processing, so as to obtain aligned sample domain data output by the sample domain adversarial module, and then optimize the feature extraction network by back-propagation gradient updating; Among them, the sample domain adversarial module includes a gradient reversal layer, a projection network and a contrast loss, and the input data passes through the gradient reversal layer, the projection network and the supervised contrast loss in sequence.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images