How to adjust the fit of analytical models to images and data

Abstract

A method for adjusting the fit of a computerized model for an imaging application first performs an application of the target domain B by a computational means using at least one image X and image analysis of the target domain B to generate a target domain B application output for X. The method then applies an application of the reference domain A by the computational means to generate a reference domain A application output for X. The method further performs a fit evaluation to generate a fit result for the at least one X. In addition, the method inspects the fit result for X and, if the inspection output is non-conforming, performs an online correction to generate a corrected application output for X.

Description

[Technical Field]

[0001] STATEMENT OF RIGHTS TO INVENTIONS UNDER FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT This work was supported in part by U.S. government grants No. 5R44NS097094-04 awarded by the NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE, No. 1R44MH121167-01 awarded by the NATIONAL INSTITUTE OF MENTAL HEALTH, and No. 1U44GM136091-04 awarded by the NATIONAL INSTITUTE OF GENERAL MEDICAL SCIENCES. The U.S. government may have certain rights in this invention.

[0002] The present invention relates to the application of machine learning and deep learning models, and more particularly to methods for tuning the fitness of computerized models for efficient deployment of image and data analysis for imaging and data processing applications. [Background technology]

[0003] a. A description of the problem that motivated the invention Imaging and data processing applications cover diverse capabilities such as speech recognition, decision support, automation, transactions, image enhancement, restoration, boundary segmentation, object detection, feature measurement, pattern recognition, and computer vision. These fields are experiencing unprecedented growth driven by new generations of computing hardware, storage, artificial intelligence (AI) software, and sensor technologies. Today, image and data analysis, embedding data flow and processing algorithms into solution pipelines for targeted applications, can be found in a wide range of industries, including manufacturing, electronics, automotive, aerospace, biomedical research, preclinical and clinical diagnostics / therapeutics, drug discovery / development and pharmaceuticals, media, telecommunications, and banking.

[0004] Machine learning (ML), particularly deep learning (DL), is a type of AI that has recently gained widespread attention. This rapid growth has been driven primarily by the invention of convolutional neural networks (CNNs), a novel machine learning algorithm architecture, and the availability of high-performance, relatively inexpensive computing units such as graphics processing units (GPUs) and temperature-tuning processing units (TPUs). CNNs have proven particularly well suited for imaging and data processing applications. In recent years, research using CNNs to analyze images and data streams has grown, underpinning fundamental changes in how imaging and data processing applications are performed and how images and data are interpreted. AI has been applied in areas such as speech recognition, AI-assisted prediction, fraud prevention, credit adjudication, speech-to-text conversion, image restoration (e.g., noise removal and resolution enhancement), label-free structure extraction from microscopy images, i.e., automated and accurate classification and segmentation of virtually stained microscopy images, biological images, and medical images, and tracking of time-lapse images. Many new generations of image and data analytics incorporate DL models for sophisticated applications. Therefore, its behavior is governed not only by the rules prescribed in the analysis, but also by the model behavior learned from the data.To complicate the problem, data inputs can be unstable and in some cases change over time.

[0005] While developing and deploying ML models is relatively quick and inexpensive, maintaining them in the field over time can be difficult and expensive. Analyses of production images and data, especially those incorporating ML models, require constant monitoring to ensure the results of applying the models are correct in development. Effective monitoring is a prerequisite for ongoing system maintenance, updates, audits, and configuration changes to ensure successful development and customer satisfaction. It is important to know whether the image and data analysis and its associated models are continuously exhibiting the expected behavior. It is also important to detect inconsistencies between the image input data and training data and the data analysis. Conformance monitoring should be designed to provide early warning for the myriad of factors that can lead to errors in the analysis of production images and data, such as data skew and model aging. Furthermore, it is highly desirable to be able to detect inconsistencies even for a single image (data point) and self-correct inconsistencies online, i.e., to ensure that correct results are generated even in inconsistent situations.

[0006] b. How has the prior art solved the problem? Monitoring relevance is straightforward when user feedback or truth-value labels are available. In this case, image and data analysis performance can be tracked and periodically reevaluated using standard metrics like precision, recall, or AUC. However, in many applications, obtaining labels is expensive (requiring manual review by human analysts) or unavailable in a timely manner. In this case, it becomes difficult to effectively monitor the internal behavior of the trained model for accuracy, but input data should be more transparent. Therefore, analyzing and comparing datasets is the first line of defense for detecting problems in situations where the world is changing in ways that could perturb an ML model.

[0007] Prior art methods monitor the model's inputs and outputs, searching for deviations outside the model's established norms. The goal is to identify changes in the ML model's behavior that contradict predictions. Given an input image or a set of feature values ​​predicted for the model output, prior art methods check whether a) the input values ​​fall within an allowed set (for categorical inputs) or allowed range (for numerical inputs), and b) the frequency of each corresponding value in the set is consistent with what has been observed in the past. This check is performed by comparing the distributions of variables in the training data to the distributions observed in the generation of these variables.

[0008] Comparisons can be performed automatically using basic statistics such as median, mean, standard deviation, maximum / minimum. For example, it is tested whether the mean is within the standard error of the mean interval. Traditionally, ad-hoc manual tests using full-scale statistical tests to compare the distributions of variables are also performed. Different tests are performed depending on the characteristics of the variables. If the variables are normally distributed, standard tests such as t-tests or ANOVAs are performed. If they are not normally distributed, non-parametric tests such as Kruskal-Wallis or Kolmogorov-Smirnov are used.

[0009] Some prior art approaches borrow signal processing techniques for time series decomposition, where time series can be used to represent sequences of model outputs or the amount of deviation between successive model runs for various types of input data, and calculate cross-correlations between identified anomalies to enable root cause analysis of model behavior.

[0010] Performing QA on complex, real-world ML applications is difficult because ML models fail for a variety of reasons that are unknown before deployment. Therefore, prior art approaches that focus on criteria and variables derived from training and validation metrics can still fail on large amounts of deployment data that are not part of the training and validation sets. Furthermore, prior art methods rely on summary statistics with variable distributions, which cannot detect anomalies in single inputs. The input / output variables used for comparison are determined ad hoc, and the variables are remotely related to the inherent behavior of the applied model in the image and data. Furthermore, when an anomaly is detected, prior art methods cannot immediately (online) correct the mismatch with respect to the affected inputs. Summary of the Invention [Problem to be solved by the invention]

[0011] a. Objectives and advantages of the invention The tuning method of the present invention not only monitors the fit of an analytical model to data, but also corrects for misfits online. This requires detection sensitivity sufficient to detect single anomalies and an adaptation method to match misfit data to the model. A first object of the present invention is to provide a method for tuning the fit of an analytical model of images and data that performs anomaly detection through self-reference. A second object of the present invention is to provide a method for tuning the fit of an analytical model of images and data that can detect single anomalies. A third object of the present invention is to provide a method for tuning the fit of an analytical model of images and data that can perform online correction. A fourth object of the present invention is to provide a method for tuning the fit of an analytical model of images and data that performs anomaly detection based on an internal representation that is pre-trained and optimized for application. A fifth object of the present invention is to provide a method for tuning the fit of an analytical model of images and data that can detect data skew and model trends over time. [Means for solving the problem]

[0012] b. How does this invention solve the problem? The present invention transforms input from a source domain into at least one reference domain and generates transformed image and data analyses for each of the reference domains. The results of applying the image and data analyses to the source input are predicted to be the same or similar to the results of applying the domain-transformed image and data analyses to the domain-transformed input. The reference domain transformation and comparison of the results allows for self-referencing without explicit labeling and truth-making of the output for a single input. Thus, the comparison can be performed on a single input rather than a population basis. This means that the sensitivity of anomaly (mismatch) detection is reduced to a single anomaly.

[0013] Domain transformation is performed by encoding input dates from the source domain into an application-optimized internal representation and then decoding from the internal representation to the desired reference domain through a trained domain generator. The application-optimized internal representation is generated by training, rather than by ad-hoc variable definition. The decoder and generator are also trained. When an anomaly is detected, a new domain transformer can be trained online specifically for the new domain to transform from the new domain to the source domain. The input data can then be transformed back to the source domain, and image and data analysis can be reapplied to the transformed data for immediate online correction. Furthermore, population-based monitoring can be performed to detect data skew and model over-time trends based on statistics derived from comparing outputs from the source domain with outputs from the reference domain. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 illustrates a process flow for a method for adjusting the suitability of a model for an imaging application in accordance with the present invention. [Figure 2]FIG. 1 illustrates a process flow for a domain transformation referencing method for imaging applications in accordance with the present invention. [Figure 3] FIG. 2 illustrates a process flow of a B-to-A domain transformer training method for imaging applications according to the present invention. [Figure 4] FIG. 2 illustrates a process flow of a BtoA image analysis transformation training method for imaging applications according to the present invention. [Figure 5] FIG. 1 illustrates a process flow for a method for adjusting the suitability of a multi-reference model for imaging applications in accordance with the present invention. [Figure 6] FIG. 1 illustrates a process flow for a method for adjusting the suitability of a model for a data processing application according to the present invention. [Figure 7] FIG. 2 illustrates a process flow of a BtoA data domain adaptive transformer training method for data processing applications according to the present invention. [Figure 8] FIG. 1 illustrates a process flow for Phase 1 domain-adaptive transformer training in accordance with the present invention. [Figure 9] FIG. 1 illustrates a process flow for Phase 2 domain-adaptive transformer training in accordance with the present invention. [Figure 10] FIG. 2 is a diagram showing the process flow of a BtoA data analysis transformation training method for data processing applications according to the present invention. [Figure 11] FIG. 1 is a schematic diagram illustrating a system configured to perform the described method. DETAILED DESCRIPTION OF THE INVENTION

[0015] The concept and preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0016] 1. How to adjust the suitability of the model for imaging applications 1 illustrates a process flow for a method of adjusting a model fit for an imaging application of the present invention. At least one image X 100 and an image analysis 102 of a target domain (referred to as "domain B") are input into an electronic storage means, such as a computer memory. An application 104 of target domain B is performed by a computational means using the at least one image X 100 and the image analysis 102 of target domain B, generating an application output 106 of target domain B for X. The computational means may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), from a local platform and / or a cloud platform and / or a mobile device. These may be implemented by electronic circuits, quantum computing devices, or optical computing devices.

[0017] A reference domain (referred to as "domain A") application 108 is performed by a computational means using at least one image X 100 to generate a reference domain A application output 110 for X. A compatibility assessment 112 is performed by a computational means using the target domain B application output 106 for X and the reference domain A application output 110 for X to generate at least one compatibility result 114 for X. Additionally, a test 116 may be performed using the at least one compatibility result 114 for X to determine an output of compatibility 118 or non-compatibility 120. If the test output is non-compatibility 120, an online correction 122 is performed to generate a corrected application output 124 for X. Individual components and / or steps are described further below.

[0018] 1.1 Input image X and target domain The input image X can be a single image or a batch of images received and applied by image analysis during development. The target domain B is the image domain in which the image analysis is designed to be performed. The model fit adjustment of the present invention ensures that the input image X fits into the target domain B, thereby enabling the image analysis to produce acceptable results.

[0019] 1.2 Image analysis The present invention is applicable to a wide range of image analysis applications, including image processing pipelines for image enhancement / restoration, boundary segmentation, object detection, feature measurement, pattern recognition, and computer vision. For example, the microscopy image analysis software Aivia provides image analysis recipes for 3D electron microscopy analysis, 3D neuron analysis, 3D object analysis, 3D object tracking, calcium oscillations, cell count, cell proliferation, cell tracking, colony analysis, exocytosis detection, filament tracking, neurite outgrowth, nucleus count, nucleus count tracking, particle tracking, phase cell tracking, pixel colocalization, and wound healing.

[0020] Furthermore, some image analyses involve machine learning processing modules such as random forests, support vector machines or DL ​​models that include multiple layers of artificial neural networks, e.g., convolutional deep neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs) and their variants, e.g., UNet, UNet with residual blocks (ResUNet), deep residual channel attention networks (RCANs), UNet with densely connected blocks (DenseUNet), conditional adversarial networks (CANs), bidirectional LSTMs, ensemble DNNs / CNNs / RNNs, hierarchical convolutional deep max-out networks, etc.

[0021] Those skilled in the art will recognize that other image processing pipelines and machine learning / DL models may also be covered by the present invention. For example, the class of DL-based image analysis includes: Mask R-CNNIt uses a region-based deep learning model for instance segmentation, called Mask R-CNN, to detect bounding boxes and also generate pixel-wise defect segmentation. Mask R-CNN is a two-stage model based on a series of deep region proposals for object detection (e.g., R-CNN , Fast R-CNN , FasterR-CNN ) is the latest in the field. In the first stage, called the region proposal network, a base CNN extracts feature maps from the input image and predicts a set of class-independent box proposals that may contain objects of interest.

[0022] 1.3 Applying Referring Domains The reference domain (Domain A) is an image domain that is significantly different from the target domain (Domain B), but within which the image analysis can be modified to operate. It can represent the maximum variation (beyond the variation in any training data) that is acceptable for the image application.

[0023] In one embodiment of the present invention, applying 108 the reference domain A is performed by a domain transformation reference method. As shown in Figure 2, the domain transformation reference method includes a BtoA domain transformation step 200 and an applying domain A step 206, where the BtoA domain transformation step 200 generates a transformed domain A image X' 204 using a BtoA domain transformer 202, and the applying domain A step 206 generates an applying domain A output 210 for X', which is the applying reference domain A output 110 for X, using image analysis 208 of the transformed domain A.

[0024] 3, the B-to-A domain transformer 202 is generated by performing B-to-A domain transformer training 302 using the image analysis 102 of the target domain B, a plurality of domain B training data 300, and at least one domain A image 304. In one embodiment of the present invention, the B-to-A domain transformer 202 is generated by the encoder EA ,E B and generator G A ,G B The details of the training procedure are described in Section 3 of this specification, and are applicable to both images and other data formats.

[0025] Note that representative domain B images from the training data for creating the target domain B image analysis are included in domain B training data 300. Only images are required for BtoA domain transformer training 302. In this case, the annotation data used for training the image analysis is not required. A single representative domain A image 304 is sufficient for domain transformer training 302. However, more domain A images can result in a more stable transformer.

[0026] The transformed domain A image analysis 208 is generated by performing BtoA image analysis transformation training 404 using a plurality of domain B training data 300 and a BtoA domain transformer 202. As shown in Figure 4, BtoA domain transformation 200 is performed using a plurality of domain B training data 300 and a BtoA domain transformer 202 to generate transformed domain A training data 400. The transformed domain A training data 400 is then used by image analysis training 402 to generate the transformed domain A image analysis 208.

[0027] 1.4 Conformity Assessment and Inspection A compatibility assessment step 112 compares the target domain B application output 106 for X with the reference domain A application output 110 for X to generate a compatibility result 114 for X. If the input image X 100 is compatible with the image analysis, the target domain B application and the reference domain A application should have the same or similar outputs 106 and 110. Therefore, even if the truth label of the target domain B application output 106 for X is unknown, compatibility assessment can be performed by examining the difference between the target domain B application output 106 for X and the reference domain A application output 110 for X. The compatibility result for X includes at least one difference metric.

[0028] When the application output is an image, error metrics such as the normalized root mean square deviation (NRMSE) and the structural similarity index (SSIM), which is a measure of the perceived similarity between two images, can be used. An alternative embodiment of fit assessment extends the image analysis to output a probability distribution that models the uncertainty of the output, and the fit of the model can then be assessed by the discrepancy between the distribution of the application output in the target domain B and the distribution of the application output in the reference domain A. Typically, Kullback-Leibler divergence can be used to measure the difference between two probability distributions, although other methods such as Jensen-Shannon divergence can be used as well.

[0029] If the output is image regions of different types or classification categories, an accuracy metric such as the F1 score can be used. Those skilled in the art will recognize that many prior art image difference metrics can be used and are within the scope of the present invention.

[0030] The fitness result for X is checked against an acceptance criterion 116, which may be determined from training data or may be dynamically definable. If the fitness result for X is within an acceptable range based on the criterion, it is determined to be conforming 118. If it is not within an acceptable range, it is determined to be non-conforming 120.

[0031] If the image X100 contains multiple samples, population-based monitoring can also be performed for data skew and for detecting trends in the model over time based on statistics derived from comparing the output from the source with the output from the reference domain. Population-based monitoring can be performed by testing basic statistics such as median, mean, standard deviation, maximum / minimum. For example, testing whether the mean value is within the standard error of the mean interval. Full-scale statistical tests can be used to compare the distribution of difference metrics. Different tests are performed depending on the characteristics of the difference metric. If the metric is normally distributed, standard tests such as t-tests or ANOVAs are performed. If it is not normally distributed, non-parametric tests such as Kruskal-Wallis or Kolmogorov-Smirnov are used.

[0032] 1.5 Online correction If the result of the fit test is a mismatch 120, the method for adjusting the fit of a computerized model of the present invention attempts to perform online correction. In the online correction step, a new domain transformer is trained and applied. The new domain transformer is trained using the image analysis 102 of the target domain B, a plurality of domain B training data 300, and the image X 100. The transformer is trained to transform the image X from the mismatched domain to the target domain B.

[0033] In one embodiment of the present invention, the encoder E A ,E B and generator G A ,G B A transformer based on the domain A is trained. As detailed in Section 3 of this specification, the transformer can be trained with a single input in the domain of image X and is bidirectional, i.e., a transformer from domain A to domain B and a transformer from domain B to domain A can be trained simultaneously.

[0034] After the transformer is generated, it is applied to image X, transforming the image to fit target domain B. Image analysis 102 of target domain B can then be applied to the transformed image X to produce a fit result. In another embodiment of the invention, the domain of non-fit of image X can be retained as an additional reference domain for online augmentation of the method for adjusting the model fit.

[0035] 2. Methods for adjusting the fit of multi-reference models for imaging applications 5 illustrates a process flow for the method of adjusting the fit of a multi-reference model for imaging applications of the present invention. At least one image X 100 and an image analysis 102 of a target domain B are input into an electronic storage means, such as a computer memory. An application 104 of the target domain B is performed by a computational means using the at least one image X 100 and the image analysis 102 of the target domain B, generating an application output 106 of the target domain B for X. The computational means may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), from a local platform and / or a cloud platform and / or a mobile device. These may be implemented by electronic circuits, quantum computing devices, or optical computing devices.

[0036] Applications 500-502 of multiple reference domains (referred to as "domains Ai") are performed by a computational means using at least one image X 100 to generate application outputs 504-506 of the reference domain Ai for X. A compatibility evaluation is performed by a computational means using the application output 106 of the target domain B for X and the application outputs 504-506 of the reference domain Ai for X to generate a compatibility result 114 for at least one X. Further, a test 116 can be performed using the compatibility result 114 for X to determine an output of compatibility 118 or non-compatibility 120. If the test output is non-compatibility 120, an online correction 122 is performed to generate a corrected application output 124 for X. Furthermore, similar to the description in Section 1.5, the online correction can be performed by training and applying a new domain transformer.

[0037] The specific components and / or steps are further described below.

[0038] 2.1 Applying Multiple Referring Domains The method of adjusting the fit of a multi-reference model allows for multiple reference domains, each of which (Domain Ai) can represent an image domain that is significantly different from the target domain (Domain B). The multi-reference approach allows a robust referencing scheme to capture the various variations that the image analysis may encounter during deployment and to modify the image analysis to operate within these domains.

[0039] In one embodiment of the present invention, the application 500-502 of the multiple reference domains Ai is performed by multiple domain transformation reference methods. Each of the domain transformation reference methods includes a BtoAi domain transformation step and a domain Ai application step. The BtoAi domain transformation step uses a BtoAi domain transformer to generate an image Xi of the transformed domain A, and the domain Ai application step uses image analysis of the transformed domain Ai to generate an application output of the domain Ai on Xi, which is an application output of the reference domain Ai on X.

[0040] The BtoAi domain transformer is generated by performing BtoAi domain transformer training using image analysis of the target domain B, training data of multiple domains B, and at least one image of domain Ai. In one embodiment of the present invention, the BtoAi domain transformer is generated by encoder E. Ai , E B and generator G Ai ,G B The training procedure is described in detail in Section 3 of this specification. The image analysis of the transformed domain Ai is generated by performing BtoAi image analysis transformation training, in which BtoAi domain transformation is performed using multiple training data of domain B and a BtoAi domain transformer to generate training data of the transformed domain Ai. The training data of the transformed domain Ai is then used by image analysis training to generate image analysis of the transformed domain Ai.

[0041] 2.2 Conformity Assessment and Inspection The compatibility assessment step 112 compares the application output 106 of the target domain B for X with the application outputs 504-506 of the reference domain Ai for X to generate a compatibility result 114 for X. If the input image X 100 is compatible with the image analysis, the application 104 of the target domain B and the application 500-502 of the reference domain Ai should have the same or similar outputs 106 and 504-506. Therefore, even if the truth label of the application output 106 of the target domain B for X is unknown, compatibility assessment can be performed by examining the application output 106 of the target domain B for X with the application outputs 504-506 of the reference domain Ai for X. The compatibility result for X includes at least one difference metric derived from the comparison.

[0042] Similar to the single-reference case (see Section 1.4), pairwise error metrics such as normalized root mean square deviation (NRMSE), structural similarity index (SSIM), Kullback-Leibler divergence, Jensen-Shannon divergence, and F1 score can be measured between the application output 106 of target domain B for X and each of the application outputs 504-506 of reference domain Ai for X. The target-to-reference metrics are referred to as inter-error metrics. Additionally, error metrics are also measured between pairs of application outputs 504-506 of reference domain Ai for X. These are referred to as intra-error metrics.

[0043] The fitness result for X is checked 116 against acceptance criteria that can be determined from training data or dynamically defined. In one embodiment of the present invention, the intra-error metric is used to dynamically define acceptance criteria for the maximum, minimum, median, and mean of the inter-error metric. If the fitness result for X is within an acceptable range based on the criteria, it is determined to be fit 118. If it is not acceptable, it is determined to be unfit 120.

[0044] If the image X100 contains multiple samples, population-based monitoring can also be performed for data skew and for detecting model trends over time based on statistics derived from comparing the output from the source with the output from the reference domain. Population-based monitoring can be performed by testing basic statistics such as median, mean, standard deviation, and maximum / minimum values. For example, a test is performed to see if the mean value is within the standard error of the mean interval. Full-scale statistical tests can be used to compare the distribution of difference metrics. Different tests are performed depending on the characteristics of the difference metric. If the metric is normally distributed, standard tests such as t-tests or ANOVAs are performed. If it is not normally distributed, non-parametric tests such as Kruskal-Wallis or Kolmogorov-Smirnov are used.

[0045] 3. How to adjust model fit for data processing applications The method for adjusting model fit is not limited to imaging applications. It can be applied to general data processing applications, and can perform domain transformations not only on images but also on non-image data. Figure 6 shows the process flow of the method for adjusting model fit for data processing applications of the present invention. At least one data X 600, an analysis 602 of a target domain (referred to as "Domain B"), a B-to-A data domain converter 610, and a transformed analysis 616 of Domain A are input into an electronic storage means such as a computer memory. An application 604 of Domain B is performed by a computing means using the at least one data X 600 and the analysis 602 of Domain B, generating an application output 606 for X. The computing means may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a processor from a local platform, a cloud platform, or a mobile device. These may be implemented by electronic circuits, quantum computing devices, or optical computing devices.

[0046] A BtoA data domain transformation 608 is performed by the computing means using at least one data X 600 and a BtoA data domain converter 610 to generate transformed domain A data X' 612. A domain A application 614 is performed by the computing means using the transformed domain A data X' 612 and a transformed domain A analysis 616 to generate an application output for X' 618. A compatibility evaluation 620 is performed by the computing means using the application output for X 606 and the application output for X' 618 to generate a compatibility result 622 for at least one X.

[0047] Additionally, a test 624 may be performed using the suitability result 622 for X to determine a suitability 626 or non-suitability 628 output. If the test output is non-suitability 628, an online correction 630 is performed to generate a suitability output 632 for a corrected X. The individual components and / or steps are further described below.

[0048] 3.1 Data Domain Adaptation Transformer Training Method A domain-adaptive transformer can map bidirectionally (A to B (AtoB) and B to A (BtoA)) between domain A, which may only be a single piece of data, and domain B, where a richer training set exists. In one embodiment of the present invention, the BtoA data domain transformer 610 is implemented using encoder E A ,E B and generator G A ,G B In addition, there are two discriminators Dr1 and Dr2 created during training in Phase 1 and Phase 2 for adaptive enhancement that optimizes the transformation for analysis. For application to the BtoA data domain transformer 610, first, E B is applied to the data in domain B, and then G A which produces the transformed domain A data.

[0049] To enrich the training samples, data is augmented for data in domain B and domain A to create P(S)[s∈B] and P(x)[x∈A]. The training involves two phases. In phase 1, an autoencoder for domain B is pre-trained in the adaptively enhanced state using the augmented data in P(S). In phase 2, adaptively enhanced autoencoders E for each of the two domains are generated. A ,G A ,E B ,G B are jointly trained using the augmented images P(S) and P(x). An autoencoder for domain B is further trained, and a separate autoencoder for domain A is initialized as a clone of the autoencoder for B pre-trained in phase 1.

[0050] As shown in Figure 7, the B to A data domain transformer 610 is generated by B to A data domain adaptive transformer training. A plurality of domain B training data 700 and domain A data 708 are input into electronic storage. Phase 1 domain adaptive transformer training 702 is performed using the domain B training data 700 and domain B analysis 602, and Phase 1 encoder E B and the generator G in phase 1 B are generated and stored as intermediate results of Phase 1 704. Phase 2, domain-adaptive transformer training 706, is performed using the training data of Domain B 700, the data of Domain A 708, the analysis of Domain B 602, and the intermediate results of Phase 1 704, to generate encoder E A ,E B and generator G A , G B A BtoA data domain transformer 610 including:

[0051] Training data for domain B The training data 700 for Domain B should include raw data and truth annotations. The sample size of the training data may be large enough to train the applied analysis. In an audio processing application of an embodiment, the data may include an audio signal, which may be a time sequence or frequency spectrum of a signal. In a microscopy image application of an embodiment, the data may include not only images and truth annotations, but also metadata such as microscope type, objective lens, excitation light source, intensity, excitation and dichroic filters, emission filters (for fluorescence microscopy), detector gain and offset, pinhole size, sampling rate / exposure time, pixel dimensions (size, time point, focus position), etc. Additionally, specimen type and state (e.g., live, fixed, organelle type, etc.) may also be stored as metadata.

[0052] Phase 1 domain-adaptive transformer training The first phase (Phase 1) of domain-adaptive transformer training involves:

number

[0053] L B+ and L D1 Phase 1 training is performed by alternately minimizing and.

[0054] In one embodiment of the present invention, the loss function L B+ is the reconstruction loss

number

number

number

number

number

[0055] The discriminator loss function is

number

[0056] The loss function above

number

[0057] The bars in the above terms indicate that the parameters of the network below the bar are fixed during backpropagation. CNNs can be configured and trained with respect to an encoder and a generator. In one embodiment of the present invention, a PatchGAN is used for the discriminator Dr1. Those skilled in the art will recognize that prior art CNN architectures (e.g., VGG, ResNet, GoogLeNet, MobileNet, etc.) can be used depending on memory, speed, and performance requirements. The training process is general-purpose and applicable to data, signals, and images.

[0058] Phase 2 domain-adaptive transformer training The second phase (Phase 2) trains an autoencoder for domain A together with a pre-trained autoencoder for domain B. A and G A is the E created in Phase 1 B and G B 9 shows the process flow of one embodiment of the domain-adaptive transformer training in Phase 2. A training image s∈P(S) 800 in the augmented domain B, where S⊂B, is initialized by the encoder E B 802 and the generator G B The training image t∈P(x) 900 [where x∈A] in the augmented domain A is generated by the encoder E A 902 and the generator G A, which creates an auto-transformed image t, referred to as t'. s' and t' are collectively referred to as the transformed image {c'} 906. Both s 800 and {c'} 906 are applied to M 808 to generate the analysis outputs of the data processing applications M(s) 810 and M({c'}) 908. A discriminator Dr 2 910 is trained to distinguish between the application outputs from the real data M(s) 810 and the application outputs from the false data M({c'}) 908. The truth of real / false 912 is known and provided for training the discriminator Dr 2 910. In one embodiment of the invention, Dr 1 814 and Dr 2 910 have the same network architecture, and at the start of Phase 2, Dr 2 is initialized to the parameter values ​​of Dr 1 trained in Phase 1. In another embodiment of the invention, a single discriminator Dr is used for Dr 1 814 and Dr 2 910. That is, a single Dr is initially trained in Phase 1 and continuously trained in Phase 2.

[0059] Phase 2 training is L AB+ and L D2 This is done by alternately minimizing

[0060] In one embodiment of the present invention, the loss function L AB+ is a weighted combination of the various loss components, i.e.

number

number

number

number

number

[0061]

number

[0062] In addition,

number

number

number

[0063] L bab-cycle and L aba-cycle is the cycle consistency loss from domain B to domain A and back to domain B, and from domain A to domain B and back to domain A. L f-cycle is the feature cycle that goes from the encoded version of sample s∈B to the sample in A and back to the latent space, i.e.,

number

[0064] The discriminator loss function is

number

[0065] As before, the bar in the above section is used to indicate that the network here is not updated during the backpropagation ("detachment") of this loss. Note that no feature loss coming from x∈A is applied. In this way, the encoder and generator for domain A are trained to adapt to the latent space of domain B, but not vice versa. Furthermore, during training, E B "Freeze" the weight of G A and E A Selective backpropagation ensures that the encoder and generator from domain A are adapted to the latent space of domain B, but not vice versa. This prevents overfitting with a single or few images from domain A. As with phase 1, the training process in phase 2 is general and applicable to any data, signal, and image.

[0066] Note that the training is bidirectional, i.e., both the BtoA converter and the AtoB converter are trained simultaneously. After training, the encoder E A ,E B and generator G A ,G B To apply the BtoA data domain converter, first create E B is applied to the data in domain B, and then G A was applied to generate the transformed domain A data. Similarly, to apply the A to B data domain transformer, we first A is applied to the domain A data, and then G B was applied to generate the transformed domain B data.

[0067] 3.2 Data analysis transformation training method The transformed domain A training data 1000 can be used to create application analyses that fit domain A. Therefore, well-established domain B training data 700 can be reused to create data processing analyses for domain A. According to one embodiment of the present invention, the training data is divided into training and validation sets for DL ​​model-based data application analysis training. The training set is used to train a deep model, and the validation set is used to evaluate the training status and readiness of the deep model. The domain B training annotation data can typically be directly used with the BtoA conversion data for the transformed domain A training data 1000. In some cases, if the annotation data is domain-dependent, the BtoA data domain converter 610 can also be trained for annotation data conversion.

[0068] 10 illustrates a process flow for the data analysis transformation training method. A plurality of domain B training data 700 is input into an electronic storage means. A B to A data domain transformation 608 is performed by a computing means using the plurality of domain B training data 700 and a B to A data domain transformer 610 to generate transformed domain A training data 1000. Analysis training 1002 is performed using the transformed domain A training data 1000 to generate an output of transformed domain A analysis 1004.

[0069] 3.3 Conformity Assessment and Inspection In a compatibility evaluation step 620, the application output for X 606 and the application output for X′ 618 are compared to generate a compatibility result for X 622. The compatibility result for X includes at least one difference metric. For image data, error metrics such as NRMSE, SSIM, Kullback-Leibler divergence, Jensen-Shannon divergence, and F1 score can be used, as described in Section 1.4. For time sequence data, a similarity measure between two time sequences can be used, such as dynamic time warping (DTW). DTW is a method for calculating the best match between two given sequences (e.g., time series) according to predetermined constraints and rules. Those skilled in the art will recognize that many prior art data difference metrics can be used and are within the scope of the present invention.

[0070] Further, an additional fitness score can be derived by applying the application output to a discriminator Dr2 910. Because Dr2 910 is trained to discriminate between application output from real data and application output from fake data, the real / false output of Dr2 can be used as an additional fitness score. If the score is high, data X 600 is predicted to be matching real data, and vice versa. The additional fitness score can be combined with at least one difference metric to generate a fitness result 622 for X.

[0071] The fitness result for X 622 is checked against an acceptance criterion that can be determined from training data or dynamically defined 624. If the fitness result for X is within an acceptable range based on the criterion, it is determined to be conforming 626. If it is not within an acceptable range, it is determined to be non-conforming 628.

[0072] If the data X contains multiple samples, population-based monitoring can also be performed for data skew, as well as for detecting trends in the model over time based on statistics derived from comparing the output from the source with the output from the transformed domain. Population-based monitoring can be performed by testing basic statistics such as median, mean, standard deviation, max / min, etc. For example, testing whether the mean value is within the standard error of the mean interval. Full-scale statistical tests can be used to compare the distribution of difference metrics. Different tests are performed depending on the characteristics of the difference metric. If the metric is normally distributed, standard tests such as t-tests or ANOVAs are performed. If it is not normally distributed, non-parametric tests such as Kruskal-Wallis or Kolmogorov-Smirnov are used.

[0073] 3.4 Online correction Similar to Section 1.5, if the result of the fit check is a misfit 628, the method for adjusting the fit of a computerized model for data processing of the present invention attempts to perform online correction. In the online correction step, a new domain transformer is trained and applied. The new domain transformer is trained using the analysis 602 of the target domain B, a plurality of domain B training data 700, and data X 600. The transformer is trained to transform data X from the misfit domain to the target domain B.

[0074] In one embodiment of the present invention, the encoder E A ,E B and generator G A ,G B A transformer based on X is trained. As detailed above in section 3.1 of the specification, such a transformer can be trained with a single input of data in the domain of X.

[0075] After the transformer is generated, it is applied to the data X 600 to transform the data to obtain a fit to the target domain B. An analysis 602 of the target domain B can then be applied to the transformed data X to produce a fitting result. In another embodiment of the invention, the data X of the misfit domain can be kept as an additional reference domain for online extension of the method for adjusting the fit of the model.

[0076] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0077] While some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step, and similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0078] Some embodiments relate to a system comprising one or more processors and one or more storage devices. inputting at least one image X and an image analysis of a target domain B into an electronic storage means; performing a target domain B application by a computational means using at least one image X and image analysis of the target domain B to generate a target domain B application output for X; performing an application of the reference domain A by a computational means using at least one image X to generate an application output of the reference domain A for X; performing a fitness assessment by a computational means using the application output of the target domain B for X and the application output of the reference domain A for X to generate at least one fitness result for X; It is structured as follows.

[0079] Further details and aspects of the system are described in relation to the proposed concepts and / or one or more of the embodiments described above or below (e.g., FIGS. 1-11). The system may include one or more additional optional features corresponding to one or more aspects of the proposed concepts and / or one or more of the embodiments described above or below. For example, the system may be a computer system as described in relation to FIG. 11.

[0080] Some embodiments relate to a system comprising one or more processors and one or more storage devices. inputting at least one image X and an analysis of the target domain B into electronic storage means; performing an adaptation of the target domain B by a computational means using at least one image X and the analysis of the target domain B to generate an adaptation output of the target domain B for X; performing an application of the plurality of reference domains Ai by a computational means using at least one image X to generate application outputs of the reference domain Ai for the plurality of X; performing a fitness assessment by a computational means using the application output of the target domain B for X and the application outputs of the reference domain Ai for the plurality of Xs to generate a fitness result for at least one X; It is structured as follows.

[0081] Further details and aspects of the system are described in relation to the proposed concepts and / or one or more of the embodiments described above or below (e.g., FIGS. 1-11). The system may include one or more additional optional features corresponding to one or more aspects of the proposed concepts and / or one or more of the embodiments described above or below. For example, the system may be a computer system as described in relation to FIG. 11.

[0082] Some embodiments relate to a system comprising one or more processors and one or more storage devices. inputting at least one data X, an analysis of domain B, a B-to-A domain converter, and an analysis of the converted domain A into electronic storage means; performing an application of Domain B by a computational means using at least one data X and the analysis of Domain B to generate an application output for X; performing a B to A data domain transformation by a computational means using at least one data X and a B to A data domain transformer to generate transformed domain A data X'; performing an application of Domain A by a computational means using the transformed Domain A data X' and the transformed Domain A analysis to generate an application output for X'; performing a fitness assessment by a computational means using the application output for X and the application output for X' to generate at least one fitness result for X; It is structured as follows.

[0083] Further details and aspects of the system are described in relation to the proposed concepts and / or one or more of the embodiments described above or below (e.g., FIGS. 1-11). The system may include one or more additional optional features corresponding to one or more aspects of the proposed concepts and / or one or more of the embodiments described above or below. For example, the system may be a computer system as described in relation to FIG. 11.

[0084] Some embodiments relate to a system for performing the methods described in connection with one or more of FIGS. 1-10 or a microscope including the system described above. Alternatively, the microscope may be part of or connected to such a system. FIG. 11 shows a schematic diagram of a system 1100 configured to perform the methods described herein. The system 1100 includes a microscope 1110 and a computer system 1120. The microscope 1110 is configured to capture images and is connected to the computer system 1120. The computer system 1120 is configured to perform at least a portion of the methods described herein. The computer system 1120 may be configured to execute machine learning algorithms. The computer system 1120 and the microscope 1110 may be separate entities or may be integrated into a common housing. The computer system 1120 may be part of a central processing system of the microscope 1110 and / or part of a subordinate component of the microscope 1110, such as a sensor, actor, camera, or lighting unit of the microscope 1110. The microscope 1110 can be used to generate one or more images used by the proposed concepts and / or one or more embodiments described above or below.

[0085] The computer system 1120 may be a local computing device (e.g., a personal computer, laptop, tablet computer, or mobile phone) with one or more processors and one or more storage devices, or may be a distributed computing system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, such as local clients and / or one or more remote server farms and / or data centers). The computer system 1120 may include any circuit or combination of circuits. In one embodiment, the computer system 1120 may include one or more processors, which may be of any type. As used herein, a processor may contemplate any type of computing circuit, such as, but not limited to, a microprocessor of a microscope or microscope component (e.g., a camera), a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit. Other types of circuits that may be included in computer system 1120 may be custom circuits, application specific integrated circuits (ASICs), etc., such as one or more circuits (e.g., communications circuits) used in wireless devices such as cell phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. Computer system 1120 may also include one or more storage devices, which may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives that handle removable media, such as compact discs (CDs), flash memory cards, digital video discs (DVDs), etc.Computer system 1120 may also include a display device, one or more speakers and a controller which may include a keyboard and / or a mouse, trackball, touch screen, voice recognition device, or any other device that allows a user of the system to input information to and receive information from computer system 1120.

[0086] Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, microprocessor, programmable computer, or electronic circuitry. In some embodiments, any one or more of the critical steps may be performed by such an apparatus.

[0087] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation may be performed by a non-transitory storage medium, such as a digital storage medium, for example, a floppy disk, a DVD, a Blu-ray, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Therefore, the digital storage medium may be computer-readable.

[0088] Some embodiments of the present disclosure include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform any of the methods described herein.

[0089] Generally, embodiments of the present invention may be implemented as a computer program product comprising program code that is operative to perform any of the methods when the computer program product is run on a computer, and that may be stored, for example, on a machine-readable carrier.

[0090] Further embodiments comprise the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0091] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein when the computer program runs on a computer.

[0092] Therefore, another embodiment of the invention is a recording medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the invention is an apparatus as described herein, comprising a processor and a recording medium.

[0093] In general, embodiments of the present disclosure can be implemented as a computer program product including program code that operates to perform any of the methods when the computer program product is executed on a computer. The program code may be stored, for example, on a machine-readable carrier. For example, the computer program may be stored on a non-transitory storage medium. Some embodiments relate to a non-transitory storage medium that includes machine-readable instructions for implementing, when executed, the proposed concepts or methods according to one or more of the above-described embodiments.

[0094] A further embodiment of the present invention is, therefore, a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.

[0095] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0096] Another embodiment comprises a computer having installed thereon the computer program for performing any of the methods described herein.

[0097] Another embodiment of the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0098] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus.

[0099] Embodiments may be based on the use of machine learning models or algorithms. Instead of relying on models and inference, machine learning may refer to algorithms and statistical models that a computer system may use to perform a particular task without using explicit instructions. For example, machine learning may use data transformations inferred from an analysis of past data and / or training data instead of rule-based data transformations. For example, image content may be analyzed using a machine learning model or algorithm. For a machine learning model to analyze image content, the machine learning model may be trained using training images as input and training content information as output. By training the machine learning model with a large number of training images and / or training sequences (e.g., words or sentences) and associated training content information (e.g., labels or annotations), the machine learning model "learns" to recognize image content, such that image content not included in the training data can be recognized using the machine learning model. The same principle may be used for other types of sensor data as well: by training the machine learning model with training sensor data and a desired output, the machine learning model "learns" a transformation between sensor data and output, which can be used to provide an output based on the non-training sensor data provided to the machine learning model. The provided data (e.g., sensor data, metadata and / or image data) may be pre-processed to obtain feature vectors that are used as input to machine learning models.

[0100] A machine learning model may be trained using training input data. The above example uses a training method called "supervised learning." In supervised learning, a machine learning model is trained using multiple training samples, where each sample may include multiple input data values ​​and multiple desired output values, i.e., each training sample is associated with a desired output value. By specifying both the training samples and the desired output value, the machine learning model "learns" during training which output value to provide based on input samples similar to the provided sample. In addition to supervised learning, semi-supervised learning may be used. In semi-supervised learning, some of the training samples lack a corresponding desired output value. Supervised learning may be based on a supervised learning algorithm (e.g., a classification algorithm, a regression algorithm, or a similarity learning algorithm). A classification algorithm may be used when the output is restricted to a limited set of values ​​(categorical variables), i.e., the input is classified into one of a limited set of values. A regression algorithm may be used when the output may have any numerical value (within a range). Similarity learning algorithms may be similar to both classification and regression algorithms, but are based on learning from examples using a similarity function that measures how similar or related two objects are. In addition to supervised or semi-supervised learning, unsupervised learning may also be used to train machine learning models. In unsupervised learning, input data may be provided (only), and unsupervised learning algorithms may be used to find structure in the input data (e.g., by grouping or clustering the input data, finding commonalities in the data). Clustering is the assignment of input data containing multiple input values ​​into multiple subsets (clusters) such that input values ​​within the same cluster are similar according to one or more (predefined) similarity criteria, but dissimilar to input values ​​contained in another cluster.

[0101] Reinforcement learning is a third group of machine learning algorithms. In other words, reinforcement learning may be used to train machine learning models. In reinforcement learning, one or more software actors (referred to as "software agents") are trained to take actions in their surroundings. Based on the actions taken, a reward is calculated. Reinforcement learning is based on training one or more software agents to select actions that increase cumulative rewards (as manifested by increasing rewards), resulting in the software agent becoming better at a given task.

[0102] Furthermore, some techniques may be applied to parts of the machine learning algorithm. For example, feature representation learning may be used. In other words, a machine learning model may be trained at least in part using feature representation learning, and / or a machine learning algorithm may include a feature representation learning component. A feature representation learning algorithm, which may be referred to as a representation learning algorithm, may not only preserve information in its input, but may also transform the information to make it useful, often as a preprocessing step before performing classification or prediction. Feature representation learning may be based on, for example, principal component analysis or cluster analysis.

[0103] In some examples, anomaly detection (i.e., outlier detection) may be used, which aims to provide identification of input values ​​that raise suspicion by differing significantly from the majority of the input or training data. In other words, a machine learning model may be trained at least in part with anomaly detection and / or a machine learning algorithm may include an anomaly detection component.

[0104] In some examples, a machine learning algorithm may use a decision tree as a predictive model. In other words, the machine learning model may be based on a decision tree. In a decision tree, an observation about an item (e.g., a set of input values) may be represented by a branch of the decision tree, and an output value corresponding to the item may be represented by a leaf of the decision tree. The decision tree may support both discrete and continuous values ​​as output values. If discrete values ​​are used, the decision tree may be represented as a classification tree, and if continuous values ​​are used, the decision tree may be represented as a regression tree.

[0105] Association rules are another technique that can be used in machine learning algorithms. In other words, a machine learning model may be based on one or more association rules. Association rules are created by identifying relationships between variables in large amounts of data. A machine learning algorithm may identify and / or utilize one or more association rules that represent knowledge derived from the data. These rules may be used, for example, to store, manipulate, or apply the knowledge.

[0106] Machine learning algorithms are typically based on machine learning models. In other words, the term "machine learning algorithm" may refer to a set of instructions that can be used to create, train, or use a machine learning model. The term "machine learning model" may refer to a data structure and / or a set of rules that represent learned knowledge (e.g., based on training performed by a machine learning algorithm). In embodiments, the use of a machine learning algorithm may refer to the use of an underlying machine learning model (or underlying machine learning models). The use of a machine learning model may refer to the machine learning model and / or the set of data structures / rules that are the machine learning model being trained by a machine learning algorithm.

[0107] For example, the machine learning model may be an artificial neural network (ANN). An ANN is a system influenced by biological neural networks, such as those found in the retina or brain. An ANN includes multiple interconnected nodes and multiple junctions, or edges, between the nodes. Typically, there are three types of nodes: input nodes that receive input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node may represent an artificial neuron. Each edge may transmit information from one node to another. The output of a node may be defined as a (nonlinear) function of its input (e.g., the sum of its inputs). The input of a node may be used in a function based on the "weights" of the edges or nodes that provide the input. The weights of the nodes and / or edges may be adjusted during the learning process. In other words, training an artificial neural network may involve adjusting the weights of the nodes and / or edges of the artificial neural network to obtain a desired output for a given input.

[0108] Alternatively, the machine learning model may be a support vector machine, a random forest model, or a gradient boosting model. A support vector machine (i.e., a support vector network) is a supervised learning model with an associated learning algorithm that can be used to analyze data (e.g., in classification or regression analysis). A support vector machine may be trained by providing input with multiple training input values ​​that belong to one of two categories. A support vector machine may be trained to assign new input values ​​to one of two categories. Alternatively, the machine learning model may be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network may use a directed acyclic graph to represent a set of random variables and their conditional dependencies. Alternatively, the machine learning model may be based on a genetic algorithm, a search algorithm and a heuristic method that mimics the process of natural selection.

[0109] The present invention has been described in considerable detail herein to provide those skilled in the art with the information necessary to apply the novel approach and to enable them to construct and use the particular components required in accordance with patent statutes and regulations. However, it should be understood that the present invention may be practiced with different equipment and devices, and that various modifications may be made, both with respect to equipment details and deep models, imaging analysis, data analysis, and operating procedures, without departing from the scope of the present invention.

Claims

1. 1. A method for adjusting the suitability of a computerized model for an imaging application by a computer system, said computer system having computing means and electronic storage means, said method comprising: a) receiving by said electronic storage means at least one image X and an image analysis of a target domain B; b) performing a Target Domain B adaptation by said computing means using said at least one image X and said Target Domain B image analysis to generate a Target Domain B adaptation output for X; c) performing an adaptation of reference domain A by said computing means using said at least one image X to generate an adaptation output of reference domain A for X; d) performing a fitness evaluation by said computing means using the application output of Target Domain B for X and the application output of Reference Domain A for X to generate at least one fitness result for X; Including, said at least one image X is a microscopic image; method.

2. The application of the reference domain A is performed by a B to A domain conversion step and a domain A application step; The method of claim 1.

3. the fitness result for X includes at least one difference metric; 3. The method according to claim 1 or 2.

4. The method comprises: e) using said at least one fitness result for X to generate a fitness or non-fit output by said computing means; f) if the output is non-conforming, performing a correction to generate a corrected application output for X by the calculation means; further comprising:

4. The method according to any one of claims 1 to 3.

5. the B to A domain transform step uses a B to A domain transformer to generate a transformed domain A image X'; The domain A application step uses the transformed domain A image analysis and the transformed domain A image X' to generate a domain A application output for X', which is a reference domain A application output for X. The method of claim 2.

6. The B-to-A domain transformer is generated by performing B-to-A domain transformer training using an image analysis of the target domain B, a plurality of domain B training data, and at least one domain A image. The method of claim 5.

7. The transformed domain A image analysis is generated by performing B to A image analysis transform training using a plurality of domain B training data and the B to A domain transformer.

7. The method according to claim 5 or 6.

8. performing the correction includes training and applying a new domain transformer to transform the at least one image X to match a target domain B; The method of claim 4.

9. 1. A method for adjusting the suitability of a plurality of computerized reference models for an imaging application by a computer system, said computer system having computing means and electronic storage means, said method comprising: a) receiving by said electronic storage means at least one image X and an image analysis of a target domain B; b) performing a Target Domain B adaptation by said computing means using said at least one image X and said Target Domain B image analysis to generate a Target Domain B adaptation output for X; c) performing an application of a plurality of reference domains Ai by said computing means using said at least one image X to generate an application output of a plurality of reference domains Ai on X; d) performing a fitness evaluation by the computing means using the application output of the target domain B for X and the application outputs of the plurality of reference domains Ai for X to generate at least one fitness result for X; Including, said at least one image X is a microscopic image; method.

10. The application of the plurality of reference domains Ai is performed by a plurality of BtoAi domain transformation steps and a plurality of domain Ai application steps; 10. The method of claim 9.

11. the fitness result for X includes at least one difference metric; 11. The method according to claim 9 or 10.

12. The method comprises: e) using said at least one fitness result for X to generate a fitness or non-fit output by said computing means; f) if the output is non-conforming, performing a correction to generate a corrected application output for X by the calculation means; further comprising:

12. The method according to any one of claims 9 to 11.

13. the B to A i domain transform step uses a B to A i domain transformer to generate a transformed domain A image Xi; The application step of the domain Ai uses the image analysis of the transformed domain Ai and the image Xi of the transformed domain A to generate an application output of the domain Ai for Xi, which is an application output of the reference domain Ai for X. The method of claim 10.

Images