Image Processing Method, Apparatus, Computer Device, and Storage Medium

Abstract

The present application relates to an image processing method, apparatus, computer device, and storage medium. The method includes: inputting a first image in a first image domain into a to-be-trained image processing model to obtain a second image in a second image domain; acquiring a first image feature corresponding to the first image and a second image feature corresponding to the second image; obtaining a first model loss value based on a target feature correlation degree between the first image feature and the second image feature; acquiring a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature; calculating a distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, and obtaining a second model loss value based on the distribution correlation degree, where the second model loss value has a negative correlation relationship with the distribution correlation degree. Using this method can improve the image processing effect. The image processing model in the present application can be an artificial intelligence-based neural network model, and an artificial intelligence cloud service is provided based on this model.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to an image processing method, apparatus, computer device, and storage medium. Background Art

[0002] With the development of artificial intelligence technology and multimedia technology, users use image information more and more frequently in daily life and production activities. For example, users can perform domain conversion on images to obtain images in different image domains. For example, a sketch image can be converted into a second-generation image.

[0003] Currently, through artificial intelligence, a machine learning model can be used to process images. The image is input into the model to obtain a processed image. However, there are often cases where the content of the processed image varies greatly from the content of the image before processing. For example, the converted image may be distorted, resulting in poor image processing effects. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide an image processing method, apparatus, computer device, and storage medium that can improve the image processing effect.

[0005] An image processing method, the method includes: obtaining a first image in a first image domain, inputting the first image into a to-be-trained image processing model to obtain a second image in a second image domain; extracting features of the first image to obtain first image features; extracting features of the second image to obtain second image features; calculating a target feature correlation degree between the first image features and the second image features, and obtaining a first model loss value based on the target feature correlation degree, where the first model loss value has a negative correlation with the target feature correlation degree; obtaining a first eigenvalue distribution corresponding to the first image features and a second eigenvalue distribution corresponding to the second image features; calculating a distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, and obtaining a second model loss value based on the distribution correlation degree, where the second model loss value has a negative correlation with the distribution correlation degree; obtaining a target model loss value based on the first model loss value and the second model loss value; adjusting model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to use the trained image processing model to process images.

[0006] An image processing device, the device comprising: an image acquisition module for acquiring a first image in a first image domain, inputting the first image into an image processing model to be trained, and obtaining a second image in a second image domain; an image feature extraction module for extracting features from the first image to obtain first image features; and extracting features from the second image to obtain second image features; a first model loss value obtaining module for calculating a target feature correlation degree between the first image features and the second image features, and obtaining a first model loss value based on the target feature correlation degree, where the first model loss value is negatively correlated with the target feature correlation degree; a feature value distribution acquisition module for acquiring a first feature value distribution corresponding to the first image features and a second feature value distribution corresponding to the second image features; a second model loss value obtaining module for calculating a distribution correlation degree between the first feature value distribution and the second feature value distribution, and obtaining a second model loss value based on the distribution correlation degree, where the second model loss value is negatively correlated with the distribution correlation degree; a target model loss value determination module for obtaining a target model loss value based on the first model loss value and the second model loss value; and a trained image processing model obtaining module for adjusting model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to process images using the trained image processing model.

[0007] In some embodiments, the feature value distribution acquisition module includes: a first statistical value obtaining unit for performing statistics on first feature values in the first image features to obtain a first statistical value corresponding to the first feature values; and a first statistical value distribution determination unit for determining a first feature value distribution corresponding to the first image features according to the first statistical value.

[0008] In some embodiments, the first statistical value includes a target mean value and a target standard deviation, and the first statistical value distribution determination unit is further configured to obtain a target coefficient set, where the target coefficient set includes a plurality of target coefficients; perform scaling processing on the target standard deviation according to the target coefficients to obtain a target scaling value; perform translation processing on the target mean value based on the target scaling value to obtain target values, and target values corresponding to each target coefficient form a target value set; determine target occurrence probabilities corresponding to the respective target values based on the target value set, and obtain a first feature value distribution according to the target occurrence probabilities corresponding to the respective target values.

[0009] In some embodiments, the target appearance probability is determined based on the probability distribution relationship corresponding to the set of target values; the first statistical value distribution determination unit is further configured to obtain a standard probability distribution, perform numerical sampling based on the standard probability distribution, and obtain a value that satisfies the standard probability distribution as the target coefficient.

[0010] In some embodiments, the target feature correlation degree includes a first feature discrimination probability, and the first model loss value obtaining module includes: a positive sample feature obtaining unit configured to splice the first image feature and the second image feature, and use the spliced feature as the positive sample feature; a first feature discrimination probability obtaining unit configured to input the positive sample feature into a feature discrimination model to obtain a first feature discrimination probability; and a first model loss value obtaining unit configured to obtain a first model loss value based on the first feature discrimination probability.

[0011] In some embodiments, the first model loss value obtaining unit is further configured to obtain a third image in a second image domain, where the third image is different from the second image; perform feature extraction on the third image to obtain a third image feature; use the target image feature and the third image feature as negative sample features, input the negative sample features into the feature discrimination model to obtain a second feature discrimination probability, where the target image feature is the first image feature or the second image feature; and obtain a first model loss value based on the first feature discrimination probability and the second feature discrimination probability, and the second feature discrimination probability is positively correlated with the first model loss value.

[0012] In some embodiments, the apparatus further includes: a model parameter adjustment gradient obtaining module configured to obtain a model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value; and a parameter adjustment module configured to adjust the model parameters of the feature discrimination model based on the model parameter adjustment gradient.

[0013] In some embodiments, the apparatus further includes: an initial image processing model obtaining module configured to obtain an initial image processing model; a current image processing model obtaining module configured to train the initial image processing model based on training images to obtain a current image processing model; a processed image obtaining module configured to input a reference image into the current image processing model for processing to obtain a processed image; and a target deformation degree determination module configured to, when it is determined that the target deformation degree of the processed image relative to the reference image is greater than a deformation degree threshold, enter the step of obtaining a first image in a first image domain.

[0014] In some embodiments, the distribution correlation calculation module includes: a distribution value acquisition unit, configured to acquire a first distribution value from the first eigenvalue distribution and acquire a second distribution value corresponding to the first distribution value from the second eigenvalue distribution; a difference value acquisition unit, configured to acquire a difference value between the first distribution value and the second distribution value; and a distribution correlation determination unit, configured to determine a distribution correlation based on the difference value, where the distribution correlation has a negative correlation with the difference value.

[0015] In some embodiments, the first image feature is obtained based on a first coding model, and the second image feature is obtained based on a second coding model. The trained image processing model obtaining module includes: a model parameter adjustment unit, configured to adjust model parameters of the image processing model, the first coding model, and the second coding model based on the target model loss value, to obtain an adjusted image processing model, first coding model, and second coding model; and a trained image processing model obtaining unit, configured to return the step of acquiring a first image in a first image domain to continue model training until the image processing model converges, to obtain a trained image processing model.

[0016] A computer device includes a memory and a processor. When the processor executes a computer program, the following steps are implemented: acquiring a first image in a first image domain, inputting the first image into a to-be-trained image processing model to obtain a second image in a second image domain; extracting features from the first image to obtain a first image feature; extracting features from the second image to obtain a second image feature; calculating a target feature correlation between the first image feature and the second image feature, and obtaining a first model loss value based on the target feature correlation, where the first model loss value has a negative correlation with the target feature correlation; acquiring a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature; calculating a distribution correlation between the first eigenvalue distribution and the second eigenvalue distribution, and obtaining a second model loss value based on the distribution correlation, where the second model loss value has a negative correlation with the distribution correlation; obtaining a target model loss value based on the first model loss value and the second model loss value; and adjusting model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to process an image by using the trained image processing model.

[0017] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the following steps are implemented: obtaining a first image in a first image domain, inputting the first image into an image processing model to be trained to obtain a second image in a second image domain; extracting features from the first image to obtain first image features; extracting features from the second image to obtain second image features; calculating a target feature correlation degree between the first image features and the second image features, and obtaining a first model loss value based on the target feature correlation degree, where the first model loss value has a negative correlation with the target feature correlation degree; obtaining a first eigenvalue distribution corresponding to the first image features and a second eigenvalue distribution corresponding to the second image features; calculating a distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, and obtaining a second model loss value based on the distribution correlation degree, where the second model loss value has a negative correlation with the distribution correlation degree; obtaining a target model loss value based on the first model loss value and the second model loss value; adjusting model parameters of the image processing model based on the target model loss value to obtain a trained image processing model for processing images using the trained image processing model.

[0018] In some embodiments, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the steps in the above method embodiments.

[0019] The above image processing method, apparatus, computer device, and storage medium obtain a first image in a first image domain, input the first image into an image processing model to be trained to obtain a second image in a second image domain, extract features from the first image to obtain first image features, extract features from the second image to obtain second image features, calculate the target feature correlation degree between the first image features and the second image features, obtain a first model loss value based on the target feature correlation degree, where the first model loss value has a negative correlation with the target feature correlation degree, obtain the first eigenvalue distribution corresponding to the first image features and the second eigenvalue distribution corresponding to the second image features, calculate the distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, obtain a second model loss value based on the distribution correlation degree, where the second model loss value has a negative correlation with the distribution correlation degree, obtain a target model loss value based on the first model loss value and the second model loss value, adjust the model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, and use the trained image processing model to process images. Since the second model loss value has a negative correlation with the distribution correlation degree, and the greater the distribution correlation degree, the less information related to the image domain in the first image features and the second image features, therefore, by adjusting the model parameters in the direction of decreasing the second model loss value, the image domain information in the first image features and the second image features can be reduced, and the content information of the image can be increased, so that the target feature correlation degree can reflect the similarity degree of the content information between the first image and the second image. Since the first model loss value has a negative correlation with the target feature correlation degree, and the greater the target feature correlation degree, the greater the similarity degree of the content information between the first image and the second image, therefore, by adjusting the model parameters in the direction of decreasing the first model loss value, the second image can be kept consistent with the first image in content information, the situation of deformation of the second image can be reduced, the accuracy of the image processing model can be improved, and the image processing effect can be improved.

[0020] An image processing method, the method comprising: obtaining an original image to be subjected to style transfer; determining a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred; determining a trained image processing model based on the first image domain and the second image domain; the image processing model being trained based on a target model loss value, the target model loss value being obtained according to a first model loss value and a second model loss value, the first model loss value being calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value being negatively correlated with the target feature correlation degree, the second model loss value being obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value being negatively correlated with the distribution correlation degree, the first image feature being an image feature corresponding to a first image in the first image domain, the first image feature being an image feature corresponding to a second image in the second image domain, the second image being generated based on the first image; inputting the original image into the trained image processing model to generate a target image in the second image domain.

[0021] An image processing apparatus, the apparatus comprising: an original image acquisition module for acquiring an original image to be subjected to style transfer; an image domain determination module for determining a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred; a trained image processing model determination module for determining a trained image processing model based on the first image domain and the second image domain; the image processing model being trained based on a target model loss value, the target model loss value being obtained according to a first model loss value and a second model loss value, the first model loss value being calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value being negatively correlated with the target feature correlation degree, the second model loss value being obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value being negatively correlated with the distribution correlation degree, the first image feature being an image feature corresponding to a first image in the first image domain, the first image feature being an image feature corresponding to a second image in the second image domain, the second image being generated based on the first image; a target image generation module for inputting the original image into the trained image processing model to generate a target image in the second image domain.

[0022] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented: obtaining an original image to be subjected to style transfer; determining a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred; determining a trained image processing model based on the first image domain and the second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value has a negative correlation with the target feature correlation degree, the second model loss value is obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value has a negative correlation with the distribution correlation degree, the first image feature is an image feature corresponding to a first image in the first image domain, the first image feature is an image feature corresponding to a second image in the second image domain, and the second image is generated based on the first image; inputting the original image into the trained image processing model to generate a target image in the second image domain.

[0023] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the following steps are implemented: obtaining an original image to be subjected to style transfer; determining a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred; determining a trained image processing model based on the first image domain and the second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value has a negative correlation with the target feature correlation degree, the second model loss value is obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value has a negative correlation with the distribution correlation degree, the first image feature is an image feature corresponding to a first image in the first image domain, the first image feature is an image feature corresponding to a second image in the second image domain, and the second image is generated based on the first image; inputting the original image into the trained image processing model to generate a target image in the second image domain.

[0024] In some embodiments, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the steps in the above method embodiments.

[0025] For the above image processing method, apparatus, computer device, and storage medium, an original image to be subjected to style transfer is obtained, a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred are determined, and a trained image processing model is determined based on the first image domain and the second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value has a negative correlation with the target feature correlation degree, the second model loss value is obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value has a negative correlation with the distribution correlation degree, the first image feature is an image feature corresponding to a first image in the first image domain, the first image feature is an image feature corresponding to a second image in the second image domain, the second image is generated based on the first image, and the original image is input into the trained image processing model to generate a target image in the second image domain. Since the second model loss value has a negative correlation with the distribution correlation degree, and the greater the distribution correlation degree, the less the information related to the image domain in the first image feature and the second image feature, therefore, by adjusting the model parameters in the direction of decreasing the second model loss value, the image domain information in the first image feature and the second image feature can be reduced, and the content information of the image can be increased, so that the target feature correlation degree can reflect the similarity degree of the content information between the first image and the second image. Since the first model loss value has a negative correlation with the target feature correlation degree, and the greater the target feature correlation degree, the greater the similarity degree of the content information between the first image and the second image, therefore, by adjusting the model parameters in the direction of decreasing the first model loss value, the second image can be made consistent with the first image in terms of content information, the situation of deformation of the second image can be reduced, the accuracy of the image processing model can be improved, and the image processing effect can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is an application environment diagram of the image processing method in some embodiments;

[0027] Figure 2 It is a flowchart of the image processing method in some embodiments;

[0028] Figure 3 Schematic diagrams of a first image and a second image in some embodiments;

[0029] Figure 4 Structural diagram of a generative adversarial network in some embodiments;

[0030] Figure 5 Structural diagram of a cyclic generative adversarial network in some embodiments;

[0031] Figure 6 Schematic diagram of the principle of the large feature discrimination probability of inputting sample features into a feature discrimination model in some embodiments;

[0032] Figure 7 Schematic flow diagram of an image processing method in some embodiments;

[0033] Figure 8 Application environment diagram of an image processing method in some embodiments;

[0034] Figure 9 Schematic flow diagram of an image processing method in some embodiments;

[0035] Figure 10 Structural block diagram of an image processing apparatus in some embodiments;

[0036] Figure 11 Structural block diagram of an image processing apparatus in some embodiments;

[0037] Figure 12 Internal structural diagram of a computer device in some embodiments;

[0038] Figure 13 Internal structural diagram of a computer device in some embodiments. Detailed implementation manners

[0039] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, a theory, method, technology and application system that perceives the environment, acquires knowledge and uses the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

[0041] Artificial intelligence technology is a comprehensive discipline that involves a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0042] Machine Learning (ML) is an interdisciplinary field that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0043] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields. For example, common ones include smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0044] The solution provided in the embodiments of this application involves technologies such as machine learning in artificial intelligence, and will be specifically described through the following embodiments:

[0045] The image processing method provided in this application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The terminal 102 can send an image domain migration request to the server 104. The image domain migration request may carry the original image, the first image domain to which the original image belongs, and the second image domain to which it is to be migrated. The server 104 can obtain a trained image processing model according to the first image domain and the second image domain. This image processing model is used to perform image domain migration on the images in the first image domain to obtain images in the second image domain. The server 104 can input the original image into this image processing model to generate a target image in the second image domain. The server 104 can return the generated target image to the terminal 102 or other terminals.

[0046] Among them, the image processing model can be trained by the server 104. Specifically, the server 104 can obtain the first image in the first image domain, input the first image into the image processing model to be trained, obtain the second image in the second image domain, extract features from the first image to obtain the first image features, extract features from the second image to obtain the second image features, calculate the target feature correlation degree between the first image features and the second image features, obtain the first model loss value based on the target feature correlation degree, and the first model loss value is negatively correlated with the target feature correlation degree. Obtain the first eigenvalue distribution corresponding to the first image features and the second eigenvalue distribution corresponding to the second image features, calculate the distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, obtain the second model loss value based on the distribution correlation degree, and the second model loss value is negatively correlated with the distribution correlation degree. Obtain the target model loss value based on the first model loss value and the second model loss value, and adjust the model parameters of the image processing model based on the target model loss value to obtain the trained image processing model, so as to use the trained image processing model to process images.

[0047] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud storage, network services, cloud communication, big data, and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.

[0048] It can be understood that the image processing model provided by the embodiments of this application can also be deployed in the terminal 102.

[0049] In some embodiments, as Figure 2 shown, an image processing method is provided. Taking the method applied to the Figure 1 server 104 as an example, the method includes the following steps:

[0050] S202, obtain the first image in the first image domain, and input the first image into the image processing model to be trained to obtain the second image in the second image domain.

[0051] Among them, the image domain refers to the domain to which the image belongs. The domain can be divided according to the image domain information. Images in the same domain have the same image domain information. The image domain information may include information for reflecting the image style, including but not limited to color or brightness. Images in different domains can be obtained by different image acquisition devices or different image acquisition personnel. For example, images acquired by one image acquisition device correspond to one domain, or images acquired with different acquisition parameters correspond to one image domain.

[0052] The first image domain and the second image domain are two different image domains respectively. The first image domain can also be called the source domain, and the second image domain can also be called the target domain. The source domain refers to the image domain to which the image belongs before being processed by the image processing model. The target domain refers to the image domain to which the image belongs after being processed by the image processing model. The first image belongs to the first image domain, and there can be multiple first images. The second image belongs to the second image domain and is generated by the image processing model according to the first image.

[0053] The image processing model is used to process images. For example, image processing can be at least one of image enhancement or image conversion. Image conversion can be cross-domain image conversion, which means converting an image from the source domain to the target domain. Cross-domain image conversion can be converting an image from one source (source domain) to an image that conforms to the standard of another source (target domain). For example, it can be image style conversion, converting an image of one style to another style. The image processing model can be a Fully Convolutional Network (FCN) model. FCN is completely composed of convolutional layers and pooling layers. Among them, cross-domain image conversion belongs to an image domain adaptation method. Image domain adaptation can also be called Domain Adaption (DA), which is a method used to solve the inconsistency of data distribution between two domains. Usually, an image or features in one domain are converted to another domain to reduce the difference between the images in the two domains.

[0054] Specifically, multiple images of the first image domain can be stored in the server. The first image can be obtained from the multiple images of the first image domain, and the first image is input into the image processing model to be trained to obtain the second image of the second image domain. The image processing model to be trained can be the model currently being trained. When the server determines that the training process of the image processing model meets the preset training method adjustment conditions, the steps of obtaining the first image of the first image domain, inputting the first image into the image processing model to be trained, and obtaining the second image of the second image domain can be executed. The preset training method adjustment conditions can include at least one of the training times reaching the preset number of training times or the image output by the image processing model meeting the preset image conditions. The preset image conditions can be, for example, that the image content information remains consistent with the image content information before processing. The image content information can be used to reflect the information of the objects in the image, including but not limited to texture, shape, or spatial relationship. The objects in the image can be inanimate objects or living objects. Inanimate objects can include furniture, such as tables or chairs. Living objects can be, for example, plants, bacteria, animals, or components of the human body. Components of the human body can be, for example, the lungs, heart, or fundus of the eye in the human body, or objects in the lungs, heart, or fundus of the eye, such as spots in the fundus or nodules in the lungs. The image content information can be, for example, the position and shape of the plant in the image. For medical images in the medical field, the image content information can be, for example, the shape of the components of the human body in the medical image, such as the shape of the "lung" in the "lung" image.

[0055] In some embodiments, the first image domain may be the image domain to which the training set for training an image detection model in the medical field belongs, and the second image domain is the image domain to which the test set for testing the image detection model belongs. The image detection model is a model for image detection and is used to obtain image detection results. The image detection results may, for example, be the detection results of objects in the image, such as including at least one of the position where the object is located or the classification result corresponding to the object. For example, the image recognition model can identify at least one of the position where the spots are located in the fundus or the category of the spots. The training set includes multiple medical images, and the test set includes multiple medical images. The training set and the test set may be collected by different medical devices and belong to different image domains. The image domains of the individual medical images in the training set are the same, the image domains of the individual medical images in the test set are the same, and the image domains of the medical images in the training set are different from those of the medical images in the test set. When training a neural network model, when the training set and the test set belong to different image domains, in order to improve the training accuracy, the images in the test set can be transformed into the image domain to which the training set belongs. For example, the first image domain may, for example, be the image domain to which the test set in the REFUGE dataset belongs, and the second image domain may, for example, be the image domain to which the training set in the REFUGE dataset belongs. The REFUGE dataset consists of 1,200 fundus images and can be used for optic disc (OD, optic disc) and optic cup (OC, Optic cup) segmentation. The training set and the test set in the REFUGE dataset are respectively collected by different fundus cameras. As Figure 3 shown, the fundus images in the first row are the images in the training set, and the fundus images in the second row are the images in the test set. It can be seen from the figure that there are obvious visual differences between the fundus images in the training set and the fundus images in the test set. The fundus images in the training set are darker, and the fundus images in the test set are brighter. It can be clearly seen that the image domains of the fundus images in the training set are different from those of the fundus images in the test set.

[0056] In some embodiments, the image processing model can be used to perform modal conversion on images, and the modality can be determined according to the attributes of the images. For example, the attributes of the images may include the image style. The image processing model can be used to convert the image from the source style to the target style. The styles of the images include at least one of cartoon style, realistic style, sketch style, comic style, or second-generation style. For example, the image processing model can be used to convert a sketch image into an image in the second-generation style.

[0057] In some embodiments, the goal of the image processing model is to reduce the change in the content of the image when processing the image, that is, it is necessary to ensure the invariance of the image content information. For example, when converting a daytime image into a nighttime image or converting an image collected by Hospital A into an image collected by Hospital B, it is necessary to ensure that the change in the content of the converted image is as small as possible to reduce the significant change in the image content information after image processing.

[0058] In some embodiments, the image processing model can be the generator network model in a Generative Adversarial Network (GAN), for example, it can be any one of the generator network models in a Cycle-Consistent Adversarial Network (CycleGAN). CycleGAN is a network framework for solving unpaired image conversion tasks. A generative adversarial network includes a generator network model and a discriminator network model. The generative adversarial network learns by having the generator network model and the discriminator network model play against each other to obtain the desired machine learning model. The goal of the generator network model is to obtain the desired output based on the input. The goal of the discriminator network model is to distinguish the output of the generator network model from real images as much as possible. The input of the discriminator network model includes the output of the generator network model and real images. The two network models learn through mutual confrontation and continuously adjust their parameters. The ultimate goal is that the generator network model should deceive the discriminator network model as much as possible so that the discriminator network model cannot determine whether the output result of the generator network model is real. The generator network model can also be called a Generator, and the discriminator network model can also be called a Discriminator. CycleGAN can include two single-direction GAN networks. The two single-direction GAN networks share the generator network model and each has a discriminator network model, that is, CycleGAN can include two generator network models and two discriminator network models. The image processing model can be, for example, Figure 4 or Figure 5 the Generator G in AB , or it can also be Figure 5 the Generator G in BA . Figure 4 the Generator G in AB and the Discriminator D B constitute a GAN network. Figure 5 the Generator G in AB , the Generator G BA , the Discriminator D A and the Discriminator D B constitute a CycleGAN network. It can be through the Generator G AB , the Generator G BA, Discriminator D A and Discriminator D B can achieve cross - domain image migration. For example, the first image Ia in the first image domain can be input into the generator G BA to obtain the second image Iab in the second image domain. The generator G AB , generator G BA , Discriminator D A and Discriminator D B 's backbone (main network) can be freely selected and tuned to the most suitable model for the data.

[0059] In some embodiments, the image processing model to be trained can be a model that has not been trained at all, or a model that has been trained and needs further optimization. For example, when the image processing model is the generator network model of CycleGAN, the traditional training method can be used to train CycleGAN. However, since the traditional training method is likely to cause the images generated by the generator network model in CycleGAN to be deformed, it is necessary to further adjust the model parameters to reduce the deformation of the generated images. Therefore, the image processing method provided in the embodiments of the present application can be used to train the model. Model parameters refer to the internal variable parameters of the model. For a neural network model, it can also be called the neural network weight.

[0060] In some embodiments, the server can first use a separate training method, such as the traditional method, to train a converged CycleGAN model, and then use the image processing method provided in the embodiments of the present application to train the generator network model in CycleGAN. The server can also use a method that combines the traditional training method and the image processing method provided in the embodiments of the present application to train CycleGAN, or use an alternating method for training. For example, first use the traditional method to train CycleGAN for the first preset number of times (such as once), and then use the joint training method provided in the embodiments of the present application to train the generator network model in CycleGAN for the second preset number of times (such as once) to complete one round of training, and then proceed to the next round of training.

[0061] S204, extract features from the first image to obtain first image features; extract features from the second image to obtain second image features.

[0062] Among them, the first image feature is the feature obtained by extracting features from the first image, and the second image feature is the feature obtained by extracting features from the second image. The image features include image content features and may also include image domain features. The image content feature is the feature corresponding to the image content information, such as a texture feature, and the image domain feature refers to the feature corresponding to the image domain information, such as a brightness feature.

[0063] Specifically, the server can extract features from the first image through the first encoding model to obtain the first image feature, and extract features from the second image using the second encoding model to obtain the second image feature. The first encoding model and the second encoding model can be obtained through joint training. For example, the model loss value can be calculated based on the first image feature output by the first encoding model and the second image feature output by the second encoding model, and the model parameters of the first encoding model and the second encoding model can be adjusted according to the model loss value. The loss value is obtained according to the loss function, and the loss function is a function used to represent the "risk" or "loss" of an event. The first encoding model and the second encoding model can be the same or different, and the encoding model can be a convolutional neural network model.

[0064] In some embodiments, when the image processing model can be the generator network model in the generative adversarial network, the first encoding model and the second encoding model can be set on the basis of the generative adversarial network, and the first encoding model and the second encoding model can be used for feature extraction. As Figure 4 shown, on the basis of the generator G AB and the discriminator D B that constitute the GAN network, the first encoding model EncA and the second encoding model EncB are set. The first encoding model EncA can encode the first image Ia into the feature space Z, represent the image using the feature vector, and obtain the first image feature Za, which can be expressed as Z a = E ncA (I a ), and the second encoding model EncB can encode the second image Iab into the feature space Z to obtain the second image feature Zab, which can be expressed as Z ab = E ncB (I ab ). Similarly, an encoding model can be added to the CycleGAN network, as Figure 5 shown.

[0065] In some embodiments, the encoding model is a model for extracting image content information. For the encoding model to be trained, the extracted image features may include both image content features and image domain features. By continuously training the encoding model, the image domain features in the extracted image features of the encoding model can be minimized, and as many image content features as possible can be included.

[0066] S206. Calculate the target feature correlation degree between the first image feature and the second image feature, and obtain a first model loss value based on the target feature correlation degree. The first model loss value is negatively correlated with the target feature correlation degree.

[0067] Among them, the target feature correlation degree is used to reflect the similarity degree between the first image feature and the second image feature. The similarity degree between the first image feature and the second image feature is positively correlated with the target feature correlation degree. The first model loss value is negatively correlated with the target feature correlation degree.

[0068] The positive correlation relationship means that: under the condition that other conditions remain unchanged, the changing directions of two variables are the same. When one variable changes from large to small, the other variable also changes from large to small. It can be understood that the positive correlation relationship here means that the changing directions are consistent, but it does not require that when one variable changes a little, the other variable must also change. For example, it can be set that when variable a is from 10 to 20, variable b is 100, and when variable a is from 20 to 30, variable b is 120. In this way, the changing directions of a and b are both that when a becomes larger, b also becomes larger. However, within the range of a from 10 to 20, b may not change.

[0069] The negative correlation relationship means that: under the condition that other conditions remain unchanged, the changing directions of two variables are opposite. When one variable changes from large to small, the other variable changes from small to large. It can be understood that the negative correlation relationship here means that the changing directions are opposite, but it does not require that when one variable changes a little, the other variable must also change.

[0070] Specifically, the server can input the first image feature and the second image feature into a feature discrimination model. For example, it can input the feature composed of the first image feature and the second image feature into the feature discrimination model. The feature discrimination model can calculate the target feature correlation degree between the first image feature and the second image feature. The feature discrimination model can be a neural network model to be trained or a trained neural network model. The input of the feature discrimination model can include positive sample features and can also include negative sample features. Positive sample features refer to the features composed of the features of the image before being processed by the image processing model and the features of the image after being processed. For example, the feature composed of the first image feature and the second image feature. Negative sample features refer to the features composed of the features of images from different image domains. For example, an image can be selected from the second image domain, feature extraction is performed on the selected image, and the extracted features are combined with the first image feature to obtain negative sample features.

[0071] In some embodiments, the feature discrimination model can calculate a probability for reflecting the magnitude of mutual information based on the input sample features. The calculated probability is positively correlated with the mutual information between the features included in the sample features. For example, when the sample feature is the positive sample feature composed of the first image feature and the second image feature, the mutual information between the first image feature and the second image feature is positively correlated with the probability obtained by the feature discrimination model. The server can obtain the target feature correlation degree based on the probability output by the feature discrimination model.

[0072] In some embodiments, when the image processing model is the generation network model in a generative adversarial network, the feature discrimination model can be set on the basis of setting an encoding model. As Figure 4 and 5 shown, the feature discrimination model can be, for example, Figure 4 or Figure 5 the DMI (Mutual Information Discriminator) in

[0073] Use the feature discrimination model to discriminate the features output by the first encoding model EncA and the second encoding model EncB, and determine the target feature correlation degree according to the discrimination result.

[0074] In some embodiments, the server may construct negative sample features corresponding to the feature discrimination model according to at least one of the first image features or the second image features, input the negative sample features into the feature discrimination model to obtain a first feature correlation degree, and calculate according to the first feature correlation degree and the target feature correlation degree to obtain a first model loss value. For example, the difference between the first feature correlation degree and the target feature correlation degree may be used as the first model loss value. The first model loss value is positively correlated with the first feature correlation degree.

[0075] In some embodiments, the target feature correlation degree is used to reflect the magnitude of the mutual information between the first image feature and the second image feature, and the mutual information between the first image feature and the second image feature is positively correlated with the target feature correlation degree. The greater the mutual information, the greater the similarity degree between the first image feature and the second image feature.

[0076] S208, obtain the first eigenvalue distribution corresponding to the first image feature and the second eigenvalue distribution corresponding to the second image feature.

[0077] Among them, the first image feature may include multiple first eigenvalues, and the second image feature may include multiple second eigenvalues. The distribution refers to the value-taking rule of the eigenvalues in the image feature. The first eigenvalue distribution includes the occurrence probabilities corresponding to multiple first distribution values, and may be a conditional probability distribution. The second eigenvalue distribution includes the occurrence probabilities corresponding to multiple second distribution values. The occurrence probability is used to represent the probability of the distribution value appearing in the eigenvalue distribution. For example, if the eigenvalue distribution is a normal distribution, the distribution value is the probability of occurrence in the normal distribution, and the occurrence probability corresponding to the distribution value can be calculated by setting the independent variable of the normal distribution function to the distribution value.

[0078] The first distribution value in the first eigenvalue distribution may be the same as or different from the first eigenvalue in the first image feature. The second distribution value in the second eigenvalue distribution may be the same as or different from the second eigenvalue in the second image feature. The number of first distribution values included in the first eigenvalue distribution is the same as the number of second distribution values included in the second eigenvalue distribution. There may be a corresponding relationship between the first distribution value and the second distribution value, for example, there is a corresponding relationship. For example, when the first distribution value and the second distribution value are calculated based on the same value, there is a corresponding relationship between the first distribution value and the second distribution value.

[0079] Specifically, the server can perform statistical calculations based on the first eigenvalue in the first image feature to obtain a first statistical value corresponding to the first image feature, and determine the first eigenvalue distribution according to the first statistical value. For example, the server can calculate the mean of each first eigenvalue to obtain the mean corresponding to the first image feature, calculate the standard deviation of each first eigenvalue to obtain the standard deviation corresponding to the first image feature, and determine the first eigenvalue distribution according to the mean and standard deviation corresponding to the first image feature.

[0080] In some embodiments, the distribution values in the eigenvalue distribution can be obtained by sampling. For example, the server can determine the distribution function satisfied by the first image feature according to the first statistical value, sample from the satisfied distribution function to obtain each sampling value, and determine the probability of each sampling value in the satisfied distribution function. According to the probabilities corresponding to each sampling value, the first eigenvalue distribution is obtained. The method for obtaining the second eigenvalue distribution can refer to the relevant steps for obtaining the first eigenvalue distribution, which will not be elaborated here.

[0081] In some embodiments, the step of determining the first eigenvalue distribution and the step of determining the second eigenvalue distribution according to the target coefficient set can be parallel, for example, they can be executed simultaneously or in sequence.

[0082] S210, calculate the distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, and obtain a second model loss value based on the distribution correlation degree. The second model loss value is negatively correlated with the distribution correlation degree.

[0083] Among them, the distribution correlation degree is used to reflect the similarity degree between the first eigenvalue distribution and the second eigenvalue distribution. The similarity degree between the first eigenvalue distribution and the second eigenvalue distribution is positively correlated with the distribution correlation degree.

[0084] Specifically, the server can select the first distribution value and the second distribution value calculated from the same target coefficient from the first eigenvalue distribution and the second eigenvalue distribution, obtain the occurrence probability corresponding to the first distribution value and the occurrence probability corresponding to the second distribution value, and obtain the distribution correlation degree according to the difference value between the occurrence probability corresponding to the first distribution value and the occurrence probability corresponding to the second distribution value.

[0085] In some embodiments, the server can use a divergence calculation formula to calculate the divergence value between the first eigenvalue distribution and the second eigenvalue distribution to obtain a target divergence, and determine the distribution correlation degree according to the target divergence. The target divergence is negatively correlated with the distribution correlation degree. The divergence calculation formula can be, for example, the KL (Kullback-Leibler) divergence. The server can obtain the second model loss value according to the target divergence. For example, the target divergence can be used as the second model loss value.

[0086] S212. Obtain a target model loss value based on the first model loss value and the second model loss value.

[0087] Among them, the target model loss value is positively correlated with the first model loss value and positively correlated with the second model loss value. The target model loss value can be the result obtained by weighted calculation of the first model loss value and the second model loss value.

[0088] Specifically, the server can select the target model loss value from the first model loss value and the second model loss value according to the loss value selection method. For example, it can select the larger one of the first model loss value and the second model loss value as the target model loss value, or select the smaller one as the target model loss value. The server can also perform weighted calculation on the first model loss value and the second model loss value to obtain the target model loss value. For example, it can calculate the result after adding the first model loss value and the second model loss value as the target model loss value.

[0089] S214. Adjust the model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to use the trained image processing model to process images.

[0090] Specifically, the gradient descent method can be used, such as the gradient descent method based on Adam, to adjust the model parameters in the image processing model in the direction of decreasing the target model loss value to obtain a trained image processing model. After obtaining the trained image processing model, the trained image processing model can be used to process images. For example, an image of the first style can be converted into an image of the second style, or an image collected by the first medical device can be converted into an image that meets the standard of an image collected by the second medical device.

[0091] It can be understood that the training of the model can be iterated multiple times, that is, the trained image processing model can be iteratively trained, and the training stops when the model convergence condition is met. The model convergence condition can be that the change in the model loss value is less than the preset loss value change, or that the change in the model parameters is less than the preset parameter change value. For example, when there are multiple first images, training can be performed multiple times, and each time multiple first images are used for model training.

[0092] In some embodiments, the server can adjust the model parameters of the image processing model, the model parameters of the first encoding model, and the parameters of the second encoding model based on the target model loss value to obtain a trained image processing model, a first encoding model, and a second encoding model, so as to use the trained image processing model to process images.

[0093] In the above image processing method, the first image in the first image domain is obtained, and the first image is input into the image processing model to be trained to obtain the second image in the second image domain. The first image is subjected to feature extraction to obtain the first image features, the second image is subjected to feature extraction to obtain the second image features, the target feature correlation degree between the first image features and the second image features is calculated, and the first model loss value is obtained based on the target feature correlation degree. The first model loss value has a negative correlation with the target feature correlation degree. The first eigenvalue distribution corresponding to the first image features and the second eigenvalue distribution corresponding to the second image features are obtained, the distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution is calculated, and the second model loss value is obtained based on the distribution correlation degree. The second model loss value has a negative correlation with the distribution correlation degree. The target model loss value is obtained based on the first model loss value and the second model loss value, and the model parameters of the image processing model are adjusted based on the target model loss value to obtain the trained image processing model, so as to use the trained image processing model to process images. Since the second model loss value has a negative correlation with the distribution correlation degree, and the greater the distribution correlation degree, the less information related to the image domain in the first image features and the second image features, therefore, by adjusting the model parameters in the direction of decreasing the second model loss value, the image domain information in the first image features and the second image features can be reduced, and the content information of the image can be increased, so that the target feature correlation degree can reflect the similarity degree of the content information between the first image and the second image. Since the first model loss value has a negative correlation with the target feature correlation degree, and the greater the target feature correlation degree, the greater the similarity degree of the content information between the first image and the second image, therefore, by adjusting the model parameters in the direction of decreasing the first model loss value, the second image can be made consistent with the first image in terms of content information, the situation of deformation of the second image can be reduced, the accuracy of the image processing model can be improved, and the image processing effect can be improved.

[0094] The principle of the image processing method proposed in this application is described as follows:

[0095] The target model loss value is used to adjust the parameters of the encoding model so that the features output by the encoding model include as little image domain information as possible and as much image content information as possible. To achieve this goal, the first model loss value and the second model loss value can be determined based on the Information Bottleneck (IB) theory. For example, when the first image features Za and the second image features Zb are obtained in Figure 4 or Figure 5 , the first model loss value and the second model loss value can be determined according to the information bottleneck theory. The information bottleneck is a theoretical basis provided to improve the robustness of representation learning, which compresses the information of the input image into a low-dimensional representation of the correlation information related to the target.

[0096] According to the information bottleneck theory, the representation form of the image content information and the representation form of the image domain information can be determined. For example, the representation form of the image content information can be I(Z ab ; I ab |I a ), and the representation form of the image domain information can be, for example, I(I a ; Z ab ). I a represents the first image, I ab represents the second image, Z a represents the first image feature, and Z ab represents the second image feature. I(Z ab ; I ab |I a ) is the conditional mutual information, indicating that under the condition of knowing the first image I a , the mutual information between the second image feature Z ab and the second image I ab can only contain redundant information other than the image content information, that is, only includes the image domain information. When I(Z ab ; I ab |I a ) becomes smaller, it means that the redundant information in the second image feature Z ab becomes smaller. I(I a ; Z ab ) represents the mutual information between the first image I a and the second image feature Z ab . Since Z ab is the feature extracted from I ab , when I(I a ; Z ab ) becomes larger, it means that the image content information in the second image feature Z ab becomes more. I(Z ab ; I ab |I a ) can be called redundant information, and I(I a ; Z ab ) can be called label information. Similarly, the representation form of the image content information can be I(Z a ; I a |I ab ), and the representation form of the image domain information can be, for example, I(I ab ; Z a ). Two loss value representation forms can be determined, which are the formulas and respectively. Among them, λ1 and λ2 represent the Lagrange multipliers for constrained optimization, both of which are non-negative numbers and can be 1 for example. and I(Zab ; I ab |I a ) is positively correlated with I(I a ; Z ab ) is negatively correlated with. By minimizing , I(Z ab ; I ab |I a ) can be minimized, and the purpose of maximizing I(I a ; Z ab ) can be achieved. is positively correlated with I(Z a ; I a |I ab ) and negatively correlated with I(I ab ; Z a ). By minimizing , I(Z a ; I a |I ab ) can be minimized, and I(I ab ; Z a ) can be maximized. Therefore, according to and , the target model loss value can be obtained, so that the features output by the encoding model include as little image domain information as possible and as much image content information as possible. Using the method based on the information bottleneck theory to solve the problem of content deformation in the domain adaptation process can learn without additional annotation information, which not only improves the convenience of model creation, but also realizes multi-directional image migration while ensuring that the image content does not deform, which is beneficial to extending to data in more scenarios.

[0097] It can be determined and 's mean value, determine the upper bound of this mean value, and minimize the problem of and , which is transformed into the problem of minimizing this upper bound. This upper bound can be expressed as:

[0098]

[0099] where D KL is the KL divergence. p(Z a |I a ) and p(Z ab |I ab ) are conditional distributions. D KL (p(Z a |I a )||p(Z ab |I ab )) and D KL (p(Zab |I ab )||p(Z a |I a )) is p(Z a |I a ) and p(Z ab |I ab ), the two KL divergences of which are used to represent the similarity between p(Z a |I a ) and p(Z ab |I ab ). The similarity between p(Z a |I a ) and p(Z ab |I ab ) is negatively correlated with the KL divergence between the two.

[0100] D KL (p(Z a |I a )||p(Z ab |I ab )) + D KL (p(Z ab |I ab )||p(Z a |I a ) aims to constrain the feature extraction of Z a and Z ab regarding the content information and exclude the redundant information of the domain. β is a hyperparameter used to adjust the weights of the two terms on the right side of the formula, which can be user-defined and can be negative. By constraining L IB to be minimized, it is possible to maximize I(Z a ; Z ab ) under the condition that the KL divergence between Z a and Z ab is minimized as much as possible, which can effectively maintain the consistency of the image content during the image conversion process. The KL divergence constrains the distributions of Z a and Z ab to make Z a and Z ab encoded by the network compressed, that is, to keep as little image domain information as possible in Z a and Z ab .

[0101] Since the probability distributions of p(Z a |I a ) and p(Z ab |I ab ) can be estimated, that is, the KL divergence between the two can be calculated, so the problem of minimizing L IB can be transformed into the problem of maximizing I(I a; Z ab ). Since I(I a ; Z ab ) has a lower bound, the problem of maximizing I(I a ; Z ab ) can be transformed into maximizing its lower bound, which is, for example, the

[0102]

[0103] in formula (4). The Donsker-Varadhan (DV) representation of mutual information using the KL divergence. DMI: Z a ×Z ab →R is a discriminator function built by a convolutional neural network. Among them, I(Z a ; Z ab ) represents the mutual information between Z a and Z ab , and I(Z a ; Z b ) represents the mutual information between Z a and Z b . D MI is the mutual information discriminator, and D MI (Z a ; Z ab ) is the discrimination result obtained by inputting Z a and Z ab into D MI . J represents the positive sample of the mutual information discriminator. For example, it can be the feature obtained by concatenating Z a and Z ab . J can also be called the joint distribution. M represents the negative sample of the mutual information discriminator. For example, it can be the feature obtained by concatenating Z a and Z b . M can also be called the marginal distribution. E M is the expectation with respect to M, and E J is the expectation with respect to J. represents the logarithm of the expectation of J minus the expectation of M.

[0104] In some embodiments, the image processing method provided by this application can be used for domain adaptation of medical images in a medical scenario. Medical images can be, for example, fundus, endoscopy, CT (Computed Tomography), MR (Magnetic Resonance) images, etc. For example, due to different devices for collecting medical images or different staining methods, the obtained medical images may vary in color or brightness, resulting in medical images including images of multiple different image domains. Through the image processing method provided by this application, images of different domains can be converted to generate images that are relatively similar between domains, so that these images can be used to train a medical model, and the trained medical model can be used in medical applications, such as for classifying or segmenting medical images.

[0105] In some embodiments, obtaining the first eigenvalue distribution corresponding to the first image feature includes: statistically calculating based on the first eigenvalues in the first image feature to obtain the first statistical value corresponding to the first eigenvalues; determining the first eigenvalue distribution corresponding to the first image feature according to the first statistical value.

[0106] Among them, the image feature may include multiple eigenvalues. The first eigenvalue refers to the eigenvalue included in the first image feature. The first statistical value is obtained by statistically calculating each first eigenvalue, and the statistical calculation includes, but is not limited to, mean calculation, standard deviation calculation, or variance calculation.

[0107] Specifically, the server can calculate the target mean by calculating the mean of each first eigenvalue, calculate the target standard deviation by calculating the standard deviation of each first eigenvalue, and obtain the first statistical value based on the target mean and the target standard deviation. The server can perform a linear operation on the target mean and the target standard deviation to obtain multiple linear operation values, determine the occurrence probability corresponding to each linear operation value, and obtain the first eigenvalue distribution according to the occurrence probability corresponding to each linear operation value.

[0108] In some embodiments, the server can obtain a set of target coefficients, and use the target coefficients in the set of target coefficients to perform a linear operation on the target mean and the target standard deviation to obtain a linear operation value. Among them, the target coefficients in the set of target coefficients can be obtained by sampling, for example, by sampling a preset distribution function. The preset distribution function can be, for example, a normal distribution function, and the normal distribution function includes a standard normal distribution and a non-standard normal distribution. The mean corresponding to the standard normal distribution is 0, and the standard deviation is 1. Of course, the set of target coefficients can be preset.

[0109] In some embodiments, the server may determine a target distribution function according to a target mean value and a target standard deviation. The independent variable of the target distribution function is set as a linear operation value, and the calculation result of the target distribution function is used as the occurrence probability corresponding to the linear operation value. Among them, the target distribution function and the preset distribution function belong to the same type of distribution function. For example, the preset distribution function is a standard normal distribution, and the target distribution function is a normal distribution function, and the mean value of this normal distribution function is the target mean value, and the standard deviation is the target standard deviation. The calculation method of the second eigenvalue distribution may refer to the calculation method of the first eigenvalue distribution, which will not be elaborated here.

[0110] In some embodiments, the steps of obtaining the first eigenvalue distribution and the steps of obtaining the second eigenvalue distribution may be carried out simultaneously. When the server obtains the target coefficient set, it may perform parallel calculations according to the target coefficient set to obtain the first eigenvalue distribution and the second eigenvalue distribution.

[0111] In some embodiments, the server may input the first image feature into a statistical operation model. The statistical operation model may calculate each first eigenvalue and output a first statistical value, such as outputting a target mean value and a target standard deviation. The server may train the statistical operation model according to the second model loss value, so that the statistical operation model can perform statistical operations accurately.

[0112] In this embodiment, since the eigenvalues in the image feature can accurately reflect the statistical information of the image feature, statistical analysis is performed according to the first eigenvalues in the first image feature to obtain the first statistical value corresponding to the first eigenvalue, and the first eigenvalue distribution corresponding to the first image feature is determined according to the first statistical value, which can improve the accuracy of the first eigenvalue distribution.

[0113] In some embodiments, the first statistical value includes a target mean value and a target standard deviation. Determining the first eigenvalue distribution corresponding to the first image feature according to the first statistical value includes: obtaining a target coefficient set, where the target coefficient set includes multiple target coefficients; performing scaling processing on the target standard deviation according to the target coefficients to obtain a target scaling value; performing translation processing on the target mean value based on the target scaling value to obtain a target value, and the target values corresponding to each target coefficient form a target value set; determining the target occurrence probability corresponding to each target value based on the target value set, and obtaining the first eigenvalue distribution according to the target occurrence probability corresponding to each target value.

[0114] Among them, the target mean value and the target standard deviation are obtained through statistical calculation based on the features in the first image feature. The target scaling value is the result obtained by scaling the target standard deviation according to the target coefficient. The target numerical value is the result obtained by translating the target mean value based on the target scaling value. The set of target numerical values includes multiple target numerical values, and the number of target numerical values is the same as the number of target coefficients, and one target coefficient corresponds to one target numerical value.

[0115] The set of target coefficients can be preset or obtained through adoption. For example, it can be the sampled values obtained by sampling the standard probability distribution. The standard probability distribution can have any distribution. For example, it can be a normal distribution. When the set of target coefficients is obtained by sampling the standard probability distribution, the data distribution of the set of target coefficients conforms to the standard probability distribution.

[0116] Specifically, the server can calculate the product of the target coefficient and the target standard deviation to implement the scaling process of the target standard deviation to obtain the target scaling value, and can perform a summation calculation on the target scaling value and the target mean value to implement the translation process of the target mean value to obtain the target numerical values corresponding to each target coefficient, and form a set of target numerical values.

[0117] In some embodiments, the server can determine the target distribution function according to the target mean value and the target standard deviation, set the independent variable of the target distribution function as the target numerical value, use the calculation result of the target distribution function as the target occurrence probability corresponding to the target numerical value, use each target numerical value as the first eigenvalue corresponding to the first eigenvalue distribution, and obtain the first eigenvalue distribution according to the target occurrence probability corresponding to each target numerical value. The first eigenvalue distribution includes the target occurrence probabilities corresponding to each target numerical value. Among them, the target distribution function and the preset distribution function belong to the same type of distribution function. For example, the preset distribution function is the standard normal distribution, and the target distribution function is the normal distribution function, and the mean value of this normal distribution function is the target mean value, and the standard deviation is the target standard deviation. Similarly, the server can calculate each second distribution value and the occurrence probability corresponding to each second distribution value in the second eigenvalue distribution by using the method provided in this embodiment. That is, one first distribution value and one second distribution value can be calculated through one target coefficient, so that there is a corresponding relationship between the first distribution value in the first eigenvalue distribution and the second distribution value in the second eigenvalue distribution.

[0118] In this embodiment, since the target mean value and the target standard deviation can accurately reflect the distribution of the features, the first eigenvalue distribution determined by the target mean value and the variance can accurately reflect the distribution of the first image feature, improving the distribution accuracy.

[0119] In some embodiments, the target appearance probability is determined based on the probability distribution relationship corresponding to the target value set; obtaining a target coefficient set, the target coefficient set including multiple target coefficients, including: obtaining a standard probability distribution, performing numerical sampling based on the standard probability distribution, and obtaining a value that satisfies the standard probability distribution as the target coefficient.

[0120] Specifically, the probability distribution relationship corresponding to the target value set refers to the distribution that the data in the target value set conforms to. The standard probability distribution can be any probability distribution, for example, it can be a standard normal distribution. The server can generate a standard data set that conforms to the standard probability distribution, randomly select from the standard data set, for example, randomly select one standard data from the standard data set at a time, and use the selected standard data as the target coefficient. The server can form the obtained target coefficients into a target coefficient set. Since the target coefficients in the target coefficient set are selected from the standard data set, the distribution of the data in the target coefficient set conforms to the standard probability distribution. For example, when the data distribution of the standard data set conforms to the standard normal distribution, the data distribution of the target coefficient set also conforms to the standard normal distribution.

[0121] In this embodiment, by performing numerical sampling based on the standard probability distribution and obtaining a value that satisfies the standard probability distribution as the target coefficient, the distribution of the data in the obtained target coefficient set can conform to the standard probability distribution, so that the target coefficient set can have regularity.

[0122] In some embodiments, the target feature correlation degree includes a first feature discrimination probability. Calculating the target feature correlation degree between the first image feature and the second image feature, and obtaining a first model loss value based on the target feature correlation degree includes: splicing the first image feature and the second image feature, and using the spliced feature as a positive sample feature; inputting the positive sample feature into a feature discrimination model to obtain a first feature discrimination probability; obtaining a first model loss value based on the first feature discrimination probability.

[0123] Among them, the positive sample feature is a concept opposite to the negative sample feature. The positive sample feature can include two or more image features, and multiple means at least three. The negative sample feature can include two or more image features. The first model loss value is negatively correlated with the first feature discrimination probability.

[0124] The first feature discrimination probability is the probability calculated by the feature discrimination model by inputting the positive sample feature obtained from the first image feature and the second image feature.

[0125] Specifically, according to the channel dimension of the convolutional network, the first image feature and the second image feature can be concatenated to obtain the corresponding positive sample feature, which includes the first image feature and the second image feature. For example, if the sizes of the first image feature and the second image feature are both 16*16*128, the positive sample feature corresponding to the first image feature and the second image feature is 16*16*256, where 16*16 is the size of the matrix output by each channel, and 128 and 256 respectively represent the number of channels.

[0126] In some embodiments, the server may use the negative value of the first feature discrimination probability as the first model loss value, or perform a logarithmic calculation on the first feature discrimination probability to obtain the first feature log probability, and use the negative value of the first feature log probability as the first model loss value. When there are multiple first images, statistical calculations, such as weighted calculations, may be performed on the first feature discrimination probability or the first feature log probability to obtain the first model loss value.

[0127] In this embodiment, the first image feature and the second image feature are concatenated, and the concatenated feature is used as the positive sample feature. The positive sample feature is input into the feature discrimination model to obtain the first feature discrimination probability, and the first model loss value is obtained based on the first feature discrimination probability. This realizes determining the first model loss value according to the positive sample feature composed of the first image feature and the second image feature, so that the model parameters of the image processing model and the feature discrimination model can be adjusted through the first model loss value, making the first feature discrimination probability change in the increasing direction, that is, in the direction of reducing the difference between the first image feature and the second image feature. In other words, it makes the image domain information included in the first image feature and the second image feature change in the decreasing direction, making the accuracy of the image processing model higher and improving the accuracy of the image processing model.

[0128] In some embodiments, obtaining the first model loss value based on the first feature discrimination probability includes: acquiring a third image of the second image domain, where the third image is different from the second image; performing feature extraction on the third image to obtain the third image feature; using the target image feature and the third image feature as the negative sample feature, and inputting the negative sample feature into the feature discrimination model to obtain the second feature discrimination probability, where the target image feature is the first image feature or the second image feature; obtaining the first model loss value based on the first feature discrimination probability and the second feature discrimination probability, and the second feature discrimination probability is positively correlated with the first model loss value.

[0129] Among them, the third image belongs to the second image domain. The third image is different from the second image. The target image feature may include at least one of the first image feature or the second image feature. The second feature discrimination probability is the probability obtained by the feature discrimination model when the negative sample feature is input into the feature discrimination model. The first model loss value is negatively correlated with the first feature discrimination probability and positively correlated with the second feature discrimination probability.

[0130] Specifically, the server can use the second encoding model to extract features from the third image to obtain the third image feature, and form the corresponding negative sample feature by combining the target image feature with the third image feature. For example, the first image feature and the third image feature can be spliced to obtain the corresponding negative sample feature.

[0131] In some embodiments, the server can obtain the first model loss value according to the difference between the second feature discrimination probability and the first feature discrimination probability. For example, the result of subtracting the first feature discrimination probability from the second feature discrimination probability can be calculated as the first model loss value, or the logarithm of the second feature discrimination probability can be calculated to obtain the second feature logarithm probability, the logarithm of the first feature discrimination probability can be calculated to obtain the first feature logarithm probability, and the difference between the second feature logarithm probability and the first feature logarithm probability can be calculated to obtain the first model loss value. When the number of positive sample features or negative sample features is multiple, multiple means at least two, and the second feature logarithm probability and the first feature logarithm probability can be statistically calculated to obtain the first model loss value.

[0132] In some embodiments, the feature discrimination model can classify the sample features according to the calculated probability to obtain the predicted category corresponding to the sample features. When the predicted category is consistent with the true category of the sample features, it indicates that the judgment result of the feature discrimination model is accurate. With the training of the feature discrimination model, the judgment result of the feature discrimination model becomes more and more accurate. The true category of the sample features can be determined according to the label of the sample features. The positive sample feature corresponds to a positive sample label, and the negative sample feature corresponds to a negative sample label. The positive sample label is, for example, "real", and the negative sample label is, for example, "fake".

[0133] Illustratively, the feature discrimination model can be, for example, Figure 6 the mutual information discriminator (DMI) in Figure 6Among them, Ib represents the third image, EncA represents the first encoding model, EncB represents the second encoding model, and Zb represents the third image feature. The positive sample feature composed of the first image feature Za and the second image feature Zab is input into the mutual information discrimination model, and the negative sample feature composed of the first image feature Za and the third image feature Zb is input into the mutual information discrimination model. J represents the positive sample feature, and M represents the negative sample feature. When the mutual information discrimination model is trained, the positive sample feature labeled as real can be accurately recognized as the real class, and the negative sample feature labeled as fake can be recognized as the fake class. The mutual information discrimination model can also be called a mutual information discriminator.

[0134] In some embodiments, the server can determine whether the sample feature is a positive sample feature or a negative sample feature according to the sample label. When it is determined that the sample label is a positive sample label, the feature discrimination probability obtained by the feature discrimination model is acquired, and the positive sample loss value is calculated according to the feature discrimination probability. The positive sample loss value is negatively correlated with the feature discrimination probability. When it is determined that the sample label is a negative sample label, the feature discrimination probability obtained by the feature discrimination model is acquired, and the negative sample loss value is calculated according to the feature discrimination probability. The negative sample loss value is positively correlated with the feature discrimination probability. The first model loss value is calculated by weighted calculation based on each positive sample loss value and each negative sample loss value.

[0135] In some embodiments, the server adjusts the parameters of the feature discrimination model and the image processing model, so that the first model loss value changes in a decreasing direction. Since the first feature discrimination probability is negatively correlated with the first model loss value, and the second feature discrimination probability is positively correlated with the first model loss value, the first feature discrimination probability can be made to change in an increasing direction, and the second feature discrimination probability can be made to change in a decreasing direction. Since the mutual information between the first image feature and the second image feature is positively correlated with the first feature discrimination probability, the mutual information between the first image feature and the second image feature can be made to change in an increasing direction, thereby realizing the maximization of the mutual information between the first image feature and the second image feature. When only the image content features are included in the first image feature and the second image feature, the maximization of the mutual information between the content features of the first image and the content features of the second image can be realized, so that the content features of the second image are consistent with the content features of the first image, reducing the deformation of the second image obtained by the image processing model, improving the accuracy of the image processing model, and improving the image processing effect.

[0136] In this embodiment, the target image feature and the third image feature are used as negative sample features, and the negative sample features are input into the feature discrimination model to obtain the second feature discrimination probability. Based on the first feature discrimination probability and the second feature discrimination probability, the first model loss value is obtained, realizing the determination of the first model loss value according to the negative sample features composed of the target image feature and the third image feature. Thus, the model parameters of the image processing model and the feature discrimination model can be adjusted through the first model loss value, enabling the feature discrimination model to accurately discriminate and making the accuracy of the image processing model higher and higher, improving the accuracy of the image processing model and the image processing effect.

[0137] In some embodiments, the method further includes: obtaining the model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value; adjusting the model parameters of the feature discrimination model based on the model parameter adjustment gradient.

[0138] Among them, the model parameter adjustment gradient refers to the parameter used to adjust the model parameters, and the model parameter adjustment gradient can be calculated based on the first model loss value. For example, the first model loss value can be differentiated to obtain the model parameter adjustment gradient.

[0139] Specifically, the server can perform backpropagation according to the first model loss value, and during the backpropagation process, update the model parameters of the feature discrimination model along the direction of the decrease of the model parameter adjustment gradient.

[0140] In some embodiments, the server can perform backpropagation according to the first model loss value, and during the backpropagation process, update the model parameters of the image processing model and the feature discrimination model along the direction of the decrease of the model parameter adjustment gradient, realizing the joint training of the image processing model and the feature discrimination model.

[0141] In this embodiment, obtaining the model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value and adjusting the model parameters of the feature discrimination model based on the model parameter adjustment gradient can quickly adjust the model parameters of the feature discrimination model, accelerate the model convergence speed, and improve the model training efficiency.

[0142] In some embodiments, the method further includes: obtaining an initial image processing model; training the initial image processing model based on training images to obtain the current image processing model; inputting a reference image into the current image processing model for processing to obtain a processed image; when it is determined that the target deformation degree of the processed image relative to the reference image is greater than the deformation degree threshold, then enter the step of obtaining the first image in the first image domain.

[0143] Among them, the initial image processing model can be a neural network model that has not been trained yet, or a model that has been trained but still needs further optimization. The training images and the reference images both belong to the first image domain, and the reference images are different from the training images. There can be multiple training images, and there can also be multiple reference images. "Multiple" means at least two. The current image processing model refers to the image processing model obtained by training at the current time. The processed image is the image obtained by processing the reference image with the current image processing model. The target deformation degree refers to the degree of deformation of the processed image relative to the reference image. The deformation degree threshold can be set as needed or can be preset.

[0144] Specifically, the server can perform multiple rounds of training on the initial image processing model with different training images. After obtaining the current image processing model in each round of training, the server can input the reference image into the current image processing model to obtain the processed image. The server can also first perform multiple rounds of training on the initial image processing model with different training images, and then use the current image processing model obtained after multiple rounds of training to process the reference image to obtain the processed image.

[0145] In some embodiments, the server can obtain the target deformation degree corresponding to the processed image from the terminal. Specifically, the server can send the processed image to one or more image viewing terminals. "Multiple" means at least two. The image viewing terminal can display the processed image and can obtain the user's operations on the processed image. For example, the image viewing terminal can display a deformation degree acquisition area, receive the deformation degree input or selected by the user through the deformation degree acquisition area, and return the deformation degree received by the deformation degree acquisition area to the server. The server can use the deformation degree returned by the image viewing terminal as the target deformation degree. When there are multiple image viewing terminals, the server can obtain the deformation degrees respectively returned by each image viewing terminal, perform statistical calculations on each deformation degree, such as weighted calculation, and use the result of the weighted calculation as the target deformation degree.

[0146] In some embodiments, the server can compare the target deformation degree with the deformation degree threshold. When it is determined that the target deformation degree is greater than the deformation degree threshold, it enters the step of obtaining the first image in the first image domain. The server can also calculate the deformation degree difference value between the target deformation degree and the deformation degree threshold. When the deformation degree difference value is greater than the difference value threshold, it enters the step of obtaining the first image in the first image domain. The deformation degree difference value can be determined as needed or can be preset.

[0147] In this embodiment, the initial image processing model is trained based on training images to obtain the current image processing model, which enables the current image processing model to learn the ability of image domain migration, that is, to learn the ability to obtain images in the second image domain from images in the first image domain. When it is determined that the target deformation degree of the processed image relative to the reference image is greater than the deformation degree threshold, the step of obtaining the first image in the first image domain is entered. That is to say, when it is determined that the target deformation degree of the processed image relative to the reference image is greater than the deformation degree threshold, the parameters of the image processing model are adjusted according to the target model loss value, which realizes the use of the target model loss value at an appropriate time, reduces the situation of deformation of the images output by the image processing model, and improves the efficiency and accuracy of model training.

[0148] In some embodiments, calculating the distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution includes: obtaining a first distribution value from the first eigenvalue distribution and obtaining a second distribution value corresponding to the first distribution value from the second eigenvalue distribution; obtaining the difference value between the first distribution value and the second distribution value; determining the distribution correlation degree based on the difference value, and the distribution correlation degree is negatively correlated with the difference value.

[0149] Among them, the first distribution value and the second distribution value calculated according to the same target coefficient have a corresponding relationship, that is, if the first distribution value is calculated by the target coefficient A, the second distribution value corresponding to the first distribution value is also calculated by the target coefficient A.

[0150] Specifically, the server can select the first distribution value and the second distribution value calculated according to the same target coefficient from the first eigenvalue distribution and the second eigenvalue distribution, calculate the difference value between the occurrence probability of the first distribution value and the occurrence probability of the second distribution value, and determine the target divergence between the first eigenvalue distribution and the second eigenvalue distribution according to the difference value. The target divergence is positively correlated with the difference value, and the distribution correlation degree is negatively correlated with the target divergence. Since the second model loss value is negatively correlated with the distribution correlation degree, the second model loss value is positively correlated with the target divergence.

[0151] In some embodiments, the target divergence includes a first divergence and a second divergence. The server may calculate the ratio between the occurrence probability of the first distribution value and the occurrence probability of the corresponding second distribution value to obtain a first probability ratio, which is used as the difference value between the first distribution value and the second distribution value. Alternatively, the server may perform calculations on the first probability ratio, such as performing a logarithmic operation on the first probability ratio, and use the result of the logarithmic operation as the difference value between the first distribution value and the second distribution value. The first divergence is determined based on this difference value, and the first divergence has a positive correlation with the first probability ratio. The server may calculate the ratio between the occurrence probability of the second distribution value and the occurrence probability of the corresponding second distribution value to obtain a second probability ratio, and determine the second divergence based on the second probability ratio. The second divergence has a positive correlation with the second probability ratio.

[0152] Since the similarity between the first eigenvalue distribution and the second eigenvalue distribution has a negative correlation with the target divergence, by adjusting the target divergence in the decreasing direction, the similarity between the first eigenvalue distribution and the second eigenvalue distribution can be adjusted in the increasing direction. Since the greater the similarity between the first eigenvalue distribution and the second eigenvalue distribution, the less image domain information included in the first image feature and the second image feature, it is possible to achieve the effect of feature extraction of the image and extract the image content features. Since the second model loss value has a positive correlation with the target divergence, by adjusting the second model loss value in the decreasing direction, it is also possible to achieve the effect that the less image domain information is included in the first image feature and the second image feature, and to achieve the effect of feature extraction of the image and extract the image content features.

[0153] In this embodiment, since the smaller the difference value, the closer the first distribution value and the second distribution value are, and the more similar the first eigenvalue distribution and the second eigenvalue distribution are, the distribution correlation degree determined based on the difference value can accurately reflect the similarity between the first eigenvalue distribution and the second eigenvalue distribution, improving the accuracy of the distribution correlation degree.

[0154] In some embodiments, the first image feature is obtained based on a first encoding model, and the second image feature is obtained based on a second encoding model. Adjusting the model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, and using the trained image processing model to process an image includes: adjusting the model parameters of the image processing model, the first encoding model, and the second encoding model based on the target model loss value to obtain an adjusted image processing model, a first encoding model, and a second encoding model; returning to the step of obtaining the first image in the first image domain to continue model training until the image processing model converges to obtain a trained image processing model.

[0155] Specifically, the server can adjust the model parameters of the image processing model, the first encoding model, and the second encoding model in the direction of reducing the target model loss value. After multiple iterations of training until the image processing model converges, a trained image processing model, a trained first encoding model, and a trained second encoding model are obtained.

[0156] In some embodiments, when the model convergence condition is satisfied, it is determined that the model converges. The model convergence condition can be that the change in the model loss value is less than a preset loss value change, or that the change in the model parameters is less than a preset parameter change value. The preset loss value change and the preset parameter change value are pre-set.

[0157] In some embodiments, the server can perform backpropagation based on the target model loss value, and during the backpropagation process, update the model parameters of the image processing model, the first encoding model, and the second encoding model along the gradient descent direction. Among them, "backward" means that the update of the parameters is opposite to the direction of image processing. The gradient descent method can be the stochastic gradient descent method or the batch gradient descent.

[0158] In some embodiments, the target model loss value includes a first model loss value and a second model loss value. The server can adjust the model parameters of the image processing model, the first encoding model, and the second encoding model in the direction of reducing the first model loss value and the direction of reducing the second model loss value, to obtain a trained image processing model, a first encoding model, and a second encoding model.

[0159] In some embodiments, the server adjusts the parameters of the feature discrimination model and the image processing model, such that the first model loss value changes in a decreasing direction. Since the first feature discrimination probability is negatively correlated with the first model loss value, and the second feature discrimination probability is positively correlated with the first model loss value, thus, the first feature discrimination probability can be made to change in an increasing direction, and the second feature discrimination probability can be made to change in a decreasing direction. Since the mutual information between the first image feature and the second image feature is positively correlated with the first feature discrimination probability, thus, the mutual information between the first image feature and the second image feature can be made to change in an increasing direction, thereby achieving the maximization of the mutual information between the first image feature and the second image feature. Since by adjusting the second model loss value in a decreasing direction, the effect of feature extraction on the image can be achieved, and the image content features can be extracted, that is, only the image content features can be included in the first image feature and the second image feature. In the case where the image features only include the image content features, achieving the maximization of the mutual information between the content features of the first image and the content features of the second image can ensure that the content features of the second image are consistent with the content features of the first image, thereby reducing the deformation of the second image obtained by the image processing model and improving the accuracy of the image processing model and the image processing effect.

[0160] In some embodiments, when the image processing model is the generator model in the generative adversarial network, the server can jointly train the feature discrimination model, the first encoding model, the second encoding model, the generator model and the discriminator model in the generative adversarial network based on the target model loss value, and iterate the training for multiple times, such as 70 times, to obtain the trained models.

[0161] In this embodiment, the joint training of the image processing model, the first encoding model and the second encoding model is realized, which improves the model accuracy and the model training efficiency, and improves the accuracy of domain adaptation.

[0162] In some embodiments, as Figure 7 shown, a method for image processing is provided. Taking the method applied to the Figure 1 terminal 102 as an example, the method includes the following steps:

[0163] S702, Obtain the original image to be subjected to style transfer.

[0164] S704, Determine the first image domain to which the original image belongs and the second image domain to which it is to be transferred.

[0165] S706. Determine a trained image processing model based on a first image domain and a second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value has a negative correlation with the target feature correlation degree, the second model loss value is obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value has a negative correlation with the distribution correlation degree, the first image feature is an image feature corresponding to a first image in the first image domain, the first image feature is an image feature corresponding to a second image in the second image domain, and the second image is generated based on the first image.

[0166] S708. Input an original image into the trained image processing model to generate a target image in the second image domain.

[0167] Among them, the original image is an image to be subjected to style transfer. The original image belongs to the first image domain. The trained image processing model is used to convert an image in the first image domain into an image in the second image domain to achieve image domain transfer. The target image is generated by the trained image processing model according to the original image, and the target image belongs to the second image domain.

[0168] In the above image processing method, an original image to be subjected to style transfer is obtained, a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred are determined, and a trained image processing model is determined based on the first image domain and the second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to a target feature correlation degree between a first image feature and a second image feature, the first model loss value is negatively correlated with the target feature correlation degree, the second model loss value is obtained according to a distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value is negatively correlated with the distribution correlation degree, the first image feature is an image feature corresponding to a first image in the first image domain, the second image feature is an image feature corresponding to a second image in the second image domain, the second image is generated based on the first image, the original image is input into the trained image processing model, and a target image in the second image domain is generated. Since the second model loss value is negatively correlated with the distribution correlation degree, and the greater the distribution correlation degree, the less the information related to the image domain in the first image feature and the second image feature, therefore, by adjusting the model parameters in the direction of decreasing the second model loss value, the image domain information in the first image feature and the second image feature can be reduced, and the content information of the image can be increased, so that the target feature correlation degree can reflect the similarity degree of the content information between the first image and the second image. Since the first model loss value is negatively correlated with the target feature correlation degree, and the greater the target feature correlation degree, the greater the similarity degree of the content information between the first image and the second image, therefore, by adjusting the model parameters in the direction of decreasing the first model loss value, the second image can be made consistent with the first image in terms of content information, the situation of deformation of the second image can be reduced, the accuracy of the image processing model can be improved, and the image processing effect can be improved.

[0169] The image processing method provided by the present application can be used to process medical images in the medical field. When training an image detection model in the medical field, the image detection model is a model for image detection and is used to obtain an image detection result. The image detection result can be, for example, a detection result of an object in the image, such as at least one of the position where the object is located or the classification result corresponding to the object. For example, the image recognition model can recognize at least one of the position where the spots are located in the fundus or the category of the spots.

[0170] Often, the images in the training set are collected by one medical device, while the medical images in the test set for testing the image detection model are collected by another medical device. Since the shooting parameters used by each device are different, it may lead to the situation that the medical images in the training set and the medical images in the test set belong to different image domains. When the image domains of the test set and the training set are different, the results obtained by using the test set to detect the accuracy of the trained image detection model are inaccurate. To solve this problem, the image domain conversion model can be trained by using the image processing method provided in this application, so that the trained image domain conversion model can convert the medical images in the test set to the image domain corresponding to the training set, so that the image domains of the converted test set and the training set are consistent, and the content in the converted medical image is the same as that in the medical image before conversion, that is, the content does not deform. For example, when the medical image is a fundus image, the fundus between the images before and after conversion remains the same. Then, the converted test set is used to test the image detection model, thereby improving the detection accuracy.

[0171] Specifically, the medical images corresponding to the test domain can be obtained to get the first image, and the first image is input into the image domain conversion model to be trained to get the second image corresponding to the training domain. The test domain refers to the image domain corresponding to the test set, and the training domain refers to the image domain corresponding to the training set. Determine the first eigenvalue distribution of the first image features corresponding to the first image, and the second eigenvalue distribution of the second image features corresponding to the second image. Adjust the model parameters of the image domain conversion model in the direction of reducing the difference between the first image features and the second image features, and in the direction of reducing the difference between the first eigenvalue distribution and the second eigenvalue distribution, to obtain the trained image domain conversion model, so that the trained image domain conversion model can convert the medical images belonging to the test set domain into medical images belonging to the training set domain, and can reduce the situation where the content in the converted medical image changes, improving the processing effect of the medical image. Thus, the trained image domain conversion model can be used to perform domain conversion on the medical images in the test set to obtain the converted test set, so that the medical images in the converted test set belong to the training image domain and the content does not deform. Using the converted test set to test the image detection model can improve the test accuracy.

[0172] The trained image processing model obtained by the image processing method provided in this application can also be used to perform image domain conversion on an image, so that the image domain of the image after the image domain conversion matches the image domain corresponding to the image detection model. For example, the image detection model is trained using the training set of the second image domain. Therefore, in order to improve the detection accuracy of the image detection model for the image to be detected, the image domain of the image to be detected from different hospitals or different medical devices can be converted into the second image domain, and then the image in the second image domain is input into the image detection model for detection, thereby improving the detection accuracy.

[0173] As Figure 8 shown, it is an application scenario diagram of the image processing method provided in some embodiments. A trained image processing model is deployed in the cloud server. The front end A802 can send an image to be subjected to image domain conversion to the cloud server 804. The cloud server can obtain the image to be subjected to image domain conversion and perform image domain conversion on the image to be subjected to image domain conversion by using the image processing method provided in this application to obtain the image after the image domain conversion. The cloud server 804 can send the image after the image domain conversion to the front end B806, or an image detection model can also be deployed in the cloud server 804. The image detection model can be used to perform image detection on the image after the image domain conversion to obtain an image detection result, and then the image detection result is sent to the front end B. The front end B can be, for example, a computer or a mobile phone, and the front end A can be an image acquisition device. It can be understood that the front end A and the front end B can be the same device or different devices.

[0174] For example, the front end A can be an image acquisition device that acquires images of the fundus of the eye. If an image detection model needs to detect images from different hospitals or different image acquisition devices, after the image acquisition device acquires the fundus image, it can send the fundus image to the cloud server. An image processing model and an image detection model are deployed on the cloud server. The image detection model is trained using the images in the second image domain, and the image processing model is trained using the image processing method of this application and is used to convert images in different image domains into the second image domain. Therefore, after the cloud server receives the fundus image, it can input the fundus image into the image processing model for image domain conversion to obtain the fundus image in the second image domain. The cloud server inputs the fundus image in the second image domain into the image detection model for detection to obtain an image detection result. The cloud server can send the image detection result to the front end B, such as the computer of a doctor or a tester. The front end B can display the image detection result.

[0175] In some embodiments, as Figure 9 shown, an image processing method is provided, including the following steps:

[0176] 902. Obtain an initial image processing model;

[0177] 904. Train the initial image processing model based on training images to obtain the current image processing model; input the reference image into the current image processing model for processing to obtain a processed image;

[0178] 906. Determine whether the target deformation degree of the processed image relative to the reference image is greater than the deformation degree threshold. If so, jump to 908; if not, return to 904.

[0179] 908. Obtain a first image in the first image domain, input the first image into the image processing model to be trained to obtain a second image in the second image domain, extract features from the first image to obtain first image features; extract features from the second image to obtain second image features; the first image features are obtained based on the first coding model, and the second image features are obtained based on the second coding model;

[0180] 910. Calculate the target feature correlation degree between the first image features and the second image features, and obtain a first model loss value based on the target feature correlation degree; Step 910 includes:

[0181] 910A. Concatenate the first image features and the second image features, use the concatenated features as positive sample features, obtain a third image in the second image domain, the third image is different from the second image, extract features from the third image to obtain third image features, and use the target image features and the third image features as negative sample features, where the target image features are the first image features or the second image features;

[0182] 910B. Input the positive sample features into the feature discrimination model to obtain a first feature discrimination probability; input the negative sample features into the feature discrimination model to obtain a second feature discrimination probability;

[0183] 910C. Obtain a first model loss value based on the first feature discrimination probability and the second feature discrimination probability, and the second feature discrimination probability is positively correlated with the first model loss value.

[0184] 912. Obtain a first eigenvalue distribution corresponding to the first image features and a second eigenvalue distribution corresponding to the second image features; calculate the distribution correlation degree between the first eigenvalue distribution and the second eigenvalue distribution, and obtain a second model loss value based on the distribution correlation degree; Step 912 includes:

[0185] 912A. Perform statistics according to the first eigenvalues in the first image features to obtain a first statistical value corresponding to the first eigenvalues; the first statistical value includes a target mean and a target standard deviation;

[0186] 912B. Obtain a standard probability distribution, perform numerical sampling based on the standard probability distribution to obtain a value that satisfies the standard probability distribution as the target coefficient;

[0187] 912C. Scale the target standard deviation according to the target coefficient to obtain a target scaling value; perform a translation process on the target mean based on the target scaling value to obtain a target value, and the target values corresponding to each target coefficient form a target value set;

[0188] 912D. Determine the target occurrence probability corresponding to each target value based on the target value set, and obtain a first eigenvalue distribution according to the target occurrence probability corresponding to each target value.

[0189] 914. Obtain a target model loss value based on the first model loss value and the second model loss value;

[0190] 916. Adjust the model parameters of the image processing model, the first coding model, and the second coding model based on the target model loss value to obtain an adjusted image processing model, first coding model, and second coding model;

[0191] 918. Determine whether the image processing model converges. If not, return to the step of obtaining the first image in the first image domain to continue model training. If so, execute 920.

[0192] 920. Obtain the trained image processing model.

[0193] It should be noted that step 910 and step 912 can be parallel.

[0194] It should be understood that although Figure 2-9 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 2-9 at least a part of the steps in

[0195] In some embodiments, such as Figure 10As shown, an image processing device is provided. This device can be a software module, a hardware module, or a combination of both, and becomes part of a computer device. Specifically, the device includes: an image acquisition module 1002, an image feature extraction module 1004, a first model loss value obtaining module 1006, a feature value distribution acquisition module 1008, a second model loss value obtaining module 1010, a target model loss value determination module 1012, and a trained image processing model obtaining module 1014, where:

[0196] The image acquisition module 1002 is configured to acquire a first image in a first image domain, input the first image into a to-be-trained image processing model, and obtain a second image in a second image domain.

[0197] The image feature extraction module 1004 is configured to extract features from the first image to obtain first image features, and extract features from the second image to obtain second image features.

[0198] The first model loss value obtaining module 1006 is configured to calculate the target feature correlation degree between the first image features and the second image features, and obtain a first model loss value based on the target feature correlation degree. The first model loss value has a negative correlation with the target feature correlation degree.

[0199] The feature value distribution acquisition module 1008 is configured to acquire a first feature value distribution corresponding to the first image features and a second feature value distribution corresponding to the second image features.

[0200] The second model loss value obtaining module 1010 is configured to calculate the distribution correlation degree between the first feature value distribution and the second feature value distribution, and obtain a second model loss value based on the distribution correlation degree. The second model loss value has a negative correlation with the distribution correlation degree.

[0201] The target model loss value determination module 1012 is configured to obtain a target model loss value based on the first model loss value and the second model loss value.

[0202] The trained image processing model obtaining module 1014 is configured to adjust the model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to process images using the trained image processing model.

[0203] In some embodiments, the feature value distribution acquisition module 1008 includes:

[0204] A first statistical value obtaining unit, configured to perform statistics according to the first feature values in the first image features to obtain a first statistical value corresponding to the first feature values.

[0205] A first statistical value distribution determination unit, configured to determine the first feature value distribution corresponding to the first image features according to the first statistical value.

[0206] In some embodiments, the first statistical value includes a target mean value and a target standard deviation. The first statistical value distribution determination unit is further configured to obtain a set of target coefficients, where the set of target coefficients includes a plurality of target coefficients; perform a scaling process on the target standard deviation according to the target coefficients to obtain a target scaling value; perform a translation process on the target mean value based on the target scaling value to obtain a target value, and the target values corresponding to the respective target coefficients form a set of target values; determine the target occurrence probabilities corresponding to the respective target values based on the set of target values, and obtain a first eigenvalue distribution according to the target occurrence probabilities corresponding to the respective target values.

[0207] In some embodiments, the target occurrence probability is determined based on the probability distribution relationship corresponding to the set of target values; the first statistical value distribution determination unit is further configured to obtain a standard probability distribution, perform numerical sampling based on the standard probability distribution, and obtain a value that satisfies the standard probability distribution as a target coefficient.

[0208] In some embodiments, the target feature correlation degree includes a first feature discrimination probability. The first model loss value obtaining module 1006 includes:

[0209] A positive sample feature obtaining unit, configured to splice the first image feature and the second image feature, and use the spliced feature as the positive sample feature.

[0210] A first feature discrimination probability obtaining unit, configured to input the positive sample feature into a feature discrimination model to obtain a first feature discrimination probability.

[0211] A first model loss value obtaining unit, configured to obtain a first model loss value based on the first feature discrimination probability.

[0212] In some embodiments, the first model loss value obtaining unit is further configured to obtain a third image in the second image domain, where the third image is different from the second image; perform feature extraction on the third image to obtain a third image feature; use the target image feature and the third image feature as negative sample features, input the negative sample features into the feature discrimination model to obtain a second feature discrimination probability, where the target image feature is the first image feature or the second image feature; obtain a first model loss value based on the first feature discrimination probability and the second feature discrimination probability, and the second feature discrimination probability is positively correlated with the first model loss value.

[0213] In some embodiments, the apparatus further includes:

[0214] A model parameter adjustment gradient obtaining module, configured to obtain a model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value.

[0215] A parameter adjustment module, configured to adjust the model parameters of the feature discrimination model based on the gradient adjusted by the model parameters.

[0216] In some embodiments, the apparatus further includes:

[0217] An initial image processing model acquisition module, configured to acquire an initial image processing model.

[0218] A current image processing model obtaining module, configured to train the initial image processing model based on training images to obtain a current image processing model.

[0219] A processed image obtaining module, configured to input a reference image into the current image processing model for processing to obtain a processed image.

[0220] A target deformation degree determination module, configured to enter the step of obtaining a first image in a first image domain when it is determined that the target deformation degree of the processed image relative to the reference image is greater than a deformation degree threshold.

[0221] In some embodiments, the second model loss value obtaining module 1010 includes:

[0222] A distribution value acquisition unit, configured to acquire a first distribution value from a first eigenvalue distribution and acquire a second distribution value corresponding to the first distribution value from a second eigenvalue distribution.

[0223] A difference value acquisition unit, configured to acquire a difference value between the first distribution value and the second distribution value.

[0224] A distribution correlation degree determination unit, configured to determine a distribution correlation degree based on the difference value, and the distribution correlation degree has a negative correlation with the difference value.

[0225] In some embodiments, the first image feature is obtained based on a first coding model, the second image feature is obtained based on a second coding model, and the trained image processing model obtaining module 1014 includes:

[0226] A model parameter adjustment unit, configured to adjust the model parameters of the image processing model, the first coding model, and the second coding model based on the target model loss value to obtain an adjusted image processing model, first coding model, and second coding model.

[0227] A trained image processing model obtaining unit, configured to return to the step of obtaining a first image in a first image domain to continue model training until the image processing model converges to obtain a trained image processing model.

[0228] In some embodiments, such as Figure 11As shown, an image processing device is provided. This device can be a software module, a hardware module, or a combination of both to form part of a computer device. Specifically, the device includes: an original image acquisition module 1102, an image domain determination module 1104, a trained image processing model determination module 1106, and a target image generation module 1108, where:

[0229] The original image acquisition module 1102 is used to acquire the original image to be subjected to style transfer.

[0230] The image domain determination module 1104 is used to determine the first image domain to which the original image belongs and the second image domain to which it is to be transferred.

[0231] The trained image processing model determination module 1106 is used to determine the trained image processing model based on the first image domain and the second image domain; the image processing model is trained based on the target model loss value, and the target model loss value is obtained from the first model loss value and the second model loss value. The first model loss value is calculated based on the target feature correlation degree between the first image feature and the second image feature, and the first model loss value is negatively correlated with the target feature correlation degree. The second model loss value is obtained from the distribution correlation degree between the first eigenvalue distribution corresponding to the first image feature and the second eigenvalue distribution corresponding to the second image feature, and the second model loss value is negatively correlated with the distribution correlation degree. The first image feature is the image feature corresponding to the first image in the first image domain, the first image feature is the image feature corresponding to the second image in the second image domain, and the second image is generated based on the first image.

[0232] The target image generation module 1108 is used to input the original image into the trained image processing model to generate the target image in the second image domain.

[0233] For the specific limitations of the image processing device, reference can be made to the limitations of the image processing method in the above text, which will not be elaborated here. Each module in the above image processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0234] In some embodiments, a computer device is provided. This computer device can be a server, and its internal structure diagram can be as Figure 12As shown in the figure. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the image processing method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an image processing method.

[0235] In some embodiments, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 13 shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image processing method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0236] Those skilled in the art can understand that Figure 12 and 13 the structures shown in the figure are only block diagrams of some parts of the structure related to the solution of this application, and do not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0237] In some embodiments, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in the above method embodiments.

[0238] In some embodiments, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0239] In some embodiments, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions that are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0240] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0241] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0242] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtaining a first image in a first image domain, and inputting the first image into an image processing model to be trained to obtain a second image in a second image domain; wherein, the image domain is a domain divided according to image domain information reflecting an image style; the content features of the first image and the content features of the second image are consistent; Performing feature extraction on the first image to obtain first image features; performing feature extraction on the second image to obtain second image features; Calculating a target feature correlation degree between the first image features and the second image features, and obtaining a first model loss value based on the target feature correlation degree, where the first model loss value has a negative correlation with the target feature correlation degree; the target feature correlation degree is used to reflect the similarity degree between the first image features and the second image features; Obtaining a first eigenvalue distribution corresponding to the first image features and a second eigenvalue distribution corresponding to the second image features; the calculation methods of the first eigenvalue distribution and the second eigenvalue distribution are the same; Obtaining a first distribution value from the first eigenvalue distribution, and obtaining a second distribution value corresponding to the first distribution value from the second eigenvalue distribution; obtaining a difference value between the first distribution value and the second distribution value; determining a distribution correlation degree based on the difference value; obtaining a second model loss value based on the distribution correlation degree, where the second model loss value has a negative correlation with the distribution correlation degree; Obtaining a target model loss value based on the first model loss value and the second model loss value; Adjusting model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to process an image by using the trained image processing model.

2. The method according to claim 1, characterized in that, The obtaining the first eigenvalue distribution corresponding to the first image features includes: Performing statistics according to first eigenvalues in the first image features to obtain a first statistical value corresponding to the first eigenvalues; Determining a first eigenvalue distribution corresponding to the first image features according to the first statistical value.

3. The method according to claim 2, characterized in that The first statistical value includes a target mean and a target standard deviation, and the determining a first eigenvalue distribution corresponding to the first image features according to the first statistical value includes: Obtaining a target coefficient set, where the target coefficient set includes a plurality of target coefficients; the target coefficients are values obtained by sampling a standard probability distribution; Performing a scaling process on the target coefficients according to the target standard deviation to obtain target scaling values; Performing a translation process on the target scaling values according to the target mean to obtain target values, and the target values corresponding to the respective target coefficients form a target value set; Determining target occurrence probabilities corresponding to the respective target values based on the target value set, obtaining a first eigenvalue distribution according to the target occurrence probabilities corresponding to the respective target values, and using the target occurrence probabilities corresponding to the respective target values as distribution values in the first eigenvalue distribution.

4. The method according to claim 3, wherein The target occurrence probability is determined based on a probability distribution relationship corresponding to the target value set; The obtaining the target coefficient set, where the target coefficient set includes a plurality of target coefficients includes: Obtain a standard probability distribution, perform numerical sampling based on the standard probability distribution, and obtain a value that satisfies the standard probability distribution as the target coefficient.

5. The method according to claim 1, wherein The target feature correlation includes a first feature discrimination probability. Calculating the target feature correlation between the first image feature and the second image feature, and obtaining a first model loss value based on the target feature correlation includes: Concatenate the first image feature and the second image feature, and use the concatenated feature as a positive sample feature; Input the positive sample feature into a feature discrimination model to obtain a first feature discrimination probability; Obtain a first model loss value based on the first feature discrimination probability.

6. The method according to claim 5, characterized in that, Obtaining the first model loss value based on the first feature discrimination probability includes: Obtain a third image in the second image domain, where the third image is different from the second image; Extract features from the third image to obtain a third image feature; Use the target image feature and the third image feature as negative sample features, input the negative sample features into a feature discrimination model to obtain a second feature discrimination probability, where the target image feature is the first image feature or the second image feature; Obtain a first model loss value based on the first feature discrimination probability and the second feature discrimination probability, and the second feature discrimination probability is positively correlated with the first model loss value.

7. The method according to claim 6, wherein The method further includes: Obtain a model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value; Adjust the model parameters of the feature discrimination model based on the model parameter adjustment gradient.

8. The method according to claim 1, characterized in that The method further includes: Obtain an initial image processing model; Train the initial image processing model based on training images to obtain a current image processing model; Input a reference image into the current image processing model for processing to obtain a processed image; When it is determined that the target deformation degree of the processed image relative to the reference image is greater than a deformation degree threshold, then enter the step of obtaining a first image in the first image domain.

9. The method according to claim 1, characterized in that, The distribution correlation is negatively correlated with the difference value.

10. The method according to claim 1, characterized in that, The first image feature is obtained based on a first coding model, and the second image feature is obtained based on a second coding model. Adjusting the model parameters of the image processing model based on the target model loss value to obtain a trained image processing model for processing images includes: Adjust the model parameters of the image processing model, the first coding model, and the second coding model based on the target model loss value to obtain an adjusted image processing model, first coding model, and second coding model; Return to the step of obtaining a first image in the first image domain to continue model training until the image processing model converges to obtain a trained image processing model.

11. An image processing method, characterized in that, The method includes: Obtain an original image to be subject to style transfer; Determine the first image domain to which the original image belongs and the second image domain to which it is to be transferred; where the image domain is a domain divided according to image domain information reflecting the image style; Determine a trained image processing model based on the first image domain and the second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to the target feature correlation degree between a first image feature and a second image feature, the first model loss value is negatively correlated with the target feature correlation degree, the second model loss value is obtained according to the distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value is negatively correlated with the distribution correlation degree, the first image feature is the image feature corresponding to the first image in the first image domain, the second image feature is the image feature corresponding to the second image in the second image domain, and the second image is generated based on the first image; the content features of the first image and the content features of the second image are consistent; the target feature correlation degree is used to reflect the similarity degree between the first image feature and the second image feature; the calculation methods of the first eigenvalue distribution and the second eigenvalue distribution are the same; the steps for determining the distribution correlation degree include: obtaining a first distribution value from the first eigenvalue distribution, and obtaining a second distribution value corresponding to the first distribution value from the second eigenvalue distribution; obtaining the difference value between the first distribution value and the second distribution value; determining the distribution correlation degree based on the difference value; Input the original image into the trained image processing model to generate a target image in the second image domain.

12. An image processing apparatus, characterized in that, The device includes: An image acquisition module, configured to acquire a first image in the first image domain, input the first image into an image processing model to be trained, and obtain a second image in the second image domain; wherein, the image domain is a domain divided according to image domain information reflecting the image style; the content features of the first image and the content features of the second image are consistent; An image feature extraction module, configured to extract features from the first image to obtain a first image feature; and extract features from the second image to obtain a second image feature; A first model loss value obtaining module, configured to calculate the target feature correlation degree between the first image feature and the second image feature, and obtain a first model loss value based on the target feature correlation degree, the first model loss value being negatively correlated with the target feature correlation degree; the target feature correlation degree is used to reflect the similarity degree between the first image feature and the second image feature; An eigenvalue distribution acquisition module, configured to acquire a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature; the calculation methods of the first eigenvalue distribution and the second eigenvalue distribution are the same; The second model loss value obtaining module is configured to obtain a first distribution value from the first eigenvalue distribution, and obtain a second distribution value corresponding to the first distribution value from the second eigenvalue distribution; obtain a difference value between the first distribution value and the second distribution value; determine a distribution correlation degree based on the difference value; and obtain a second model loss value based on the distribution correlation degree, where the second model loss value has a negative correlation with the distribution correlation degree. The target model loss value determining module is configured to obtain a target model loss value based on the first model loss value and the second model loss value. The trained image processing model obtaining module is configured to adjust model parameters of the image processing model based on the target model loss value to obtain a trained image processing model, so as to process an image by using the trained image processing model.

13. The image processing apparatus according to claim 12, wherein The eigenvalue distribution obtaining module includes: The first statistic value obtaining unit is configured to perform statistics on the first eigenvalue in the first image feature to obtain a first statistic value corresponding to the first eigenvalue. The first statistic value distribution determining unit is configured to determine a first eigenvalue distribution corresponding to the first image feature according to the first statistic value.

14. The image processing apparatus according to claim 13, wherein The first statistic value includes a target mean and a target standard deviation. The first statistic value distribution determining unit is further configured to obtain a target coefficient set, where the target coefficient set includes a plurality of target coefficients; the target coefficients are values obtained by sampling a standard probability distribution; perform a scaling process on the target coefficients according to the target standard deviation to obtain target scaling values; perform a translation process on the target scaling values based on the target mean to obtain target values, and the target values corresponding to the respective target coefficients form a target value set. Determine a target occurrence probability corresponding to each of the target values based on the target value set, obtain a first eigenvalue distribution according to the target occurrence probabilities corresponding to the respective target values, and use the target occurrence probabilities corresponding to the respective target values as distribution values in the first eigenvalue distribution.

15. The image processing apparatus according to claim 14, wherein The target occurrence probability is determined based on a probability distribution relationship corresponding to the target value set; the first statistic value distribution determining unit is further configured to obtain a standard probability distribution, and perform numerical sampling based on the standard probability distribution to obtain values satisfying the standard probability distribution as target coefficients.

16. The image processing apparatus according to claim 12, wherein The target feature correlation degree includes a first feature discrimination probability. The first model loss value obtaining module includes: The positive sample feature obtaining unit is configured to splice the first image feature and the second image feature, and use the spliced feature as a positive sample feature. The first feature discrimination probability obtaining unit is configured to input the positive sample feature into a feature discrimination model to obtain a first feature discrimination probability. The first model loss value obtaining unit is configured to obtain a first model loss value based on the first feature discrimination probability.

17. The image processing apparatus according to claim 16, wherein The first model loss value obtaining unit is further configured to obtain a third image in a second image domain, where the third image is different from the second image; extract features from the third image to obtain third image features; use the target image features and the third image features as negative sample features, input the negative sample features into a feature discrimination model to obtain a second feature discrimination probability, where the target image features are the first image features or the second image features; and obtain a first model loss value based on the first feature discrimination probability and the second feature discrimination probability, and the second feature discrimination probability is positively correlated with the first model loss value.

18. The image processing apparatus according to claim 17, wherein The apparatus further includes: A model parameter adjustment gradient obtaining module, configured to obtain a model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value; A parameter adjustment module, configured to adjust the model parameters of the feature discrimination model based on the model parameter adjustment gradient.

19. The image processing apparatus according to claim 12, wherein The apparatus further includes: An initial image processing model obtaining module, configured to obtain an initial image processing model; A current image processing model obtaining module, configured to train the initial image processing model based on training images to obtain a current image processing model; A processed image obtaining module, configured to input a reference image into the current image processing model for processing to obtain a processed image; A target deformation degree determining module, configured to, when it is determined that the target deformation degree of the processed image relative to the reference image is greater than a deformation degree threshold, enter the step of obtaining a first image in a first image domain.

20. The image processing apparatus according to claim 12, wherein The distribution correlation degree is negatively correlated with the difference value.

21. The image processing apparatus according to claim 12, wherein The first image features are obtained based on a first encoding model, and the second image features are obtained based on a second encoding model. The trained image processing model obtaining module includes: A model parameter adjustment unit, configured to adjust the model parameters of the image processing model, the first encoding model, and the second encoding model based on the target model loss value to obtain an adjusted image processing model, a first encoding model, and a second encoding model; A trained image processing model obtaining unit, configured to return to the step of obtaining a first image in a first image domain to continue model training until the image processing model converges to obtain a trained image processing model.

22. An image processing apparatus, characterized in that, The apparatus includes: An original image obtaining module, configured to obtain an original image to be subjected to style transfer; An image domain determining module, configured to determine a first image domain to which the original image belongs and a second image domain to which the original image is to be transferred; where the image domain is a domain divided according to image domain information reflecting the image style. A trained image processing model determination module for determining a trained image processing model based on the first image domain and the second image domain; the image processing model is trained based on a target model loss value, the target model loss value is obtained according to a first model loss value and a second model loss value, the first model loss value is calculated according to the target feature correlation degree between a first image feature and a second image feature, the first model loss value is negatively correlated with the target feature correlation degree, the second model loss value is obtained according to the distribution correlation degree between a first eigenvalue distribution corresponding to the first image feature and a second eigenvalue distribution corresponding to the second image feature, the second model loss value is negatively correlated with the distribution correlation degree, the first image feature is the image feature corresponding to the first image in the first image domain, the second image feature is the image feature corresponding to the second image in the second image domain, the second image is generated based on the first image; the content features of the first image and the content features of the second image are consistent; the target feature correlation degree is used to reflect the similarity degree between the first image feature and the second image feature; the calculation methods of the first eigenvalue distribution and the second eigenvalue distribution are the same; the steps for determining the distribution correlation degree include: obtaining a first distribution value from the first eigenvalue distribution, and obtaining a second distribution value corresponding to the first distribution value from the second eigenvalue distribution; obtaining the difference value between the first distribution value and the second distribution value; determining the distribution correlation degree based on the difference value; A target image generation module for inputting the original image into the trained image processing model to generate a target image in the second image domain.

23. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.

24. A computer-readable storage medium stores a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 11 is implemented.

25. A computer program product comprising computer instructions, characterized in that, When the computer instruction is executed by the processor, the method according to any one of claims 1 to 11 is implemented.

Images