Model training method and device, image processing method and device and electronic equipment

Abstract

The invention is suitable for the technical field of image processing, and provides a model training method, an image processing method and device, and electronic equipment, and the method comprises the steps: obtaining a plurality of source domain sample images corresponding to a source domain; at least one source domain sample image is processed through a pre-trained color prompter, a corresponding target domain sample image is obtained, the color prompter is used for converting the source domain sample image into an image with color distribution conforming to a target domain, the target domain sample image is obtained, and the target domain is different from the source domain; and training an image processing model according to each source domain sample image and at least one target domain sample image to obtain a pre-trained image processing model. The generalization ability of the image processing model can be improved.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to model training methods, image processing methods, apparatus, electronic devices and computer-readable storage media. Background Technology

[0002] With the development of artificial intelligence technology, image processing tasks can be widely applied in fields such as smart security, autonomous driving, and smart healthcare.

[0003] Image processing tasks typically include various tasks such as image recognition, image classification, and image segmentation. In image processing, trained models are usually used to process the images to be processed. However, in practical applications, due to the limitations of the training scenario, the model can usually only handle data from a single training scenario well. When processing data from scenarios different from the training scenario, the model's accuracy will significantly decrease, affecting the image processing results. Summary of the Invention

[0004] This application provides a model training method, an image processing method, an apparatus, and an electronic device, which can improve the generalization ability of image processing models.

[0005] In a first aspect, embodiments of this application provide a model training method, including:

[0006] Obtain multiple source domain sample images corresponding to the source domain;

[0007] At least one source domain sample image is processed by a pre-trained color cue to obtain a corresponding target domain sample image. The color cue is used to convert the source domain sample image into an image whose color distribution conforms to the target domain to obtain the target domain sample image. The target domain is different from the source domain.

[0008] The image processing model is trained based on each of the source domain sample images and at least one of the target domain sample images to obtain the pre-trained image processing model.

[0009] In one possible implementation of the first aspect, the color cue includes a resampling network, a parameter prediction network, and a transformation network. The step of processing at least one source domain sample image using the pre-trained color cue to obtain a corresponding target domain sample image includes:

[0010] For any one of the source domain sample images to be processed:

[0011] The source domain sample image is transformed by the resampling network to obtain a resampled image. The transformation process is used to convert the source domain sample image from the initial color space to the target color space.

[0012] The parameter prediction network is used to analyze the color values ​​of each pixel in the resampled image to obtain the conversion parameters.

[0013] The target domain sample image is determined based on the resampled image and the transformation parameters.

[0014] In one possible implementation of the first aspect, the transformation parameters include weights and biases for the mean, and weights and biases for the variance, wherein determining the target domain sample image based on the resampled image and the transformation parameters includes:

[0015] The first mean and the first variance are determined based on the color values ​​of each pixel in the resampled image;

[0016] The predicted mean is determined based on the first mean, the weight of the mean, and the bias.

[0017] The prediction variance is determined based on the first variance, the weights of the variances, and the bias.

[0018] The target domain sample image is determined based on the resampled image, the predicted mean, and the predicted variance.

[0019] In one possible implementation of the first aspect, the number of resampled images is greater than 1, and before analyzing the color values ​​of each pixel in the resampled images through the parameter prediction network to obtain the transformation parameters, the method further includes:

[0020] For any of the resampled images, calculate the mean and variance of the color values ​​of each pixel in the resampled image to obtain the second mean and the corresponding second variance of the resampled image;

[0021] Calculate the mean and variance of each of the second means to obtain the third mean and the corresponding third variance;

[0022] Calculate the mean and variance of each of the second variances to obtain the fourth mean and the corresponding fourth variance;

[0023] The resampled images are normalized based on each of the second means, each of the second variances, the third means, each of the third variances, each of the second variances, the fourth means, and each of the fourth variances to obtain the normalized resampled images.

[0024] Correspondingly, the step of analyzing the color values ​​of each pixel in the resampled image through the parameter prediction network to obtain the transformation parameters includes:

[0025] The conversion parameters are obtained by analyzing the color values ​​of each pixel in the normalized resampled image through the parameter prediction network.

[0026] In one possible implementation of the first aspect, the color cue is trained through the following steps:

[0027] Obtain the target domain image corresponding to the target domain;

[0028] The target domain image is transformed using the resampling network to obtain the resampled image;

[0029] The conversion parameters are obtained by analyzing the parameter prediction network and the color values ​​of each pixel in the resampled image.

[0030] A predicted image is determined based on the resampled image and the transformation parameters, wherein the predicted image is an image in the initial color space;

[0031] The parameters of the parameter prediction network are updated based on the difference between the target domain image and the predicted image to obtain the updated parameter prediction network.

[0032] The pre-trained color cue is determined based on the updated parameters of the prediction network.

[0033] In one possible implementation of the first aspect, processing at least one source domain sample image using a pre-trained color cue to obtain a target domain sample image includes:

[0034] At least one source domain sample image is processed by multiple color cues to obtain a corresponding target domain sample image, wherein different color cues correspond to different target domains.

[0035] Secondly, embodiments of this application provide an image processing method, including:

[0036] Obtain the image to be processed;

[0037] The image to be processed is input into a pre-trained image processing model to obtain the image processing result output by the image processing model, wherein the image processing model is trained according to the model training method described in any of the first aspects above.

[0038] Thirdly, embodiments of this application provide a model training apparatus, including:

[0039] The source domain sample acquisition module is used to acquire multiple source domain sample images corresponding to the source domain;

[0040] The target domain sample acquisition module is used to process at least one of the source domain sample images through a pre-trained color cue to obtain a corresponding target domain sample image. The color cue is used to convert the source domain sample image into an image whose color distribution conforms to the target domain to obtain the target domain sample image. The target domain is different from the source domain.

[0041] The training module is used to train the image processing model based on each of the source domain sample images and at least one of the target domain sample images to obtain the pre-trained image processing model.

[0042] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the model training method described in the first aspect or the image processing method described in the second aspect.

[0043] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the model training method described in the first aspect or the image processing method described in the second aspect.

[0044] In a sixth aspect, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the model training method described in the first aspect or the image processing method described in the second aspect.

[0045] The beneficial effects of the embodiments in this application compared with the prior art are:

[0046] In this embodiment, before training the image processing model, at least one source domain sample image is processed using a color cue to obtain a target domain sample image. Since the color cue can convert the source domain sample image into an image whose color distribution conforms to the target domain (i.e., the color distribution of the obtained target domain sample image conforms to the color distribution characteristics of the target domain), training the image processing model using both the source domain sample image and the converted target domain sample image enables the trained image processing model to have better domain generalization ability, thereby improving the accuracy of the image processing model and thus improving the image processing effect. Furthermore, since training does not require acquiring and labeling the target domain image, the labor and labeling costs required for collecting target domain sample images are effectively reduced, which is beneficial for practical applications. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0048] Figure 1 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of a process for determining a target domain sample image provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of another process for determining a target domain sample image provided in an embodiment of this application;

[0051] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application;

[0053] Figure 6 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application;

[0054] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0055] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0056] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0057] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0060] Example 1:

[0061] Figure 1 A schematic flowchart of a model training method provided in an embodiment of this application is shown, and is described in detail below:

[0062] Step S101: Obtain multiple source domain sample images corresponding to the source domain.

[0063] The source domains mentioned above are determined based on the actual application scenarios of the image processing model.

[0064] It is understandable that the source domain can refer to a specific environment, while the source domain sample image is a labeled sample image obtained based on the source domain.

[0065] For example, when applying an image processing model to a pedestrian tracking task, the source domain can be the outdoor environment of the park during the day. In this case, the source domain sample images can be images containing human bodies taken outdoors in the park during the day. Alternatively, the source domain can refer to a specific camera. In this case, the source domain sample images can be images containing human bodies taken by that specific camera.

[0066] For example, suppose the image processing module is applied to the human detection task in the scenario of a vehicle reversing. In this case, the source domain can be underground parking garages and outdoor scenes under daytime lighting conditions. In this case, images containing human bodies can be captured by the vehicle's reversing camera in underground parking garages and various outdoor scenes during the day and used as source domain sample images.

[0067] In other embodiments, multiple source domain sample images corresponding to the source domain can be obtained directly from a public dataset that matches the source domain, without the need to take additional images and perform annotations, thereby reducing the cost of training image processing models.

[0068] Step S102: At least one of the above source domain sample images is processed by a pre-trained color cue to obtain a corresponding target domain sample image. The color cue is used to convert the above source domain sample image into an image whose color distribution conforms to the target domain, thereby obtaining the above target domain sample image. The above target domain is different from the above source domain.

[0069] Optionally, the aforementioned color cue can be constructed based on network structures such as attention-based convolutional neural networks, generative adversarial networks, or U-Net (Convolutional Networks for Biomedical Image Segmentation). The specific structure can be set according to the actual application scenario. This application embodiment does not impose specific restrictions on the network structure of the color cue.

[0070] To improve the accuracy of the image processing model, before training the model using source domain sample images corresponding to the source domain, a pre-trained color cue is used to convert at least one source domain sample image into an image whose target color distribution conforms to the target domain, thus obtaining the target domain sample image. Since the target domain is different from the source domain, the above processing allows direct acquisition of the sample image corresponding to the target domain. This means that the training data for the image processing model can be augmented without the need for additional collection and annotation of target domain images, effectively reducing the labor and annotation costs required for collecting target domain sample images, which is beneficial for practical applications.

[0071] Step S103: Train the image processing model based on each of the above-mentioned source domain sample images and at least one of the above-mentioned target domain sample images to obtain the pre-trained image processing model.

[0072] Optionally, the image processing model described above can be constructed based on model structures such as convolutional neural networks, recurrent neural networks, fully connected neural networks, or transformer models. The appropriate model structure can be selected to construct the required image processing model according to the actual application scenario.

[0073] As an example, when the above image processing model is used for pedestrian re-identification tasks, it can be constructed based on the Vision Transformer (VIT) model, which is based on a transformer structure. The VIT model can capture the processing dependencies in the image from a global perspective, thereby improving the performance of the image processing model in visual tasks such as pedestrian re-identification.

[0074] As another example, when image processing models are applied to segmentation tasks, they can be built based on deep residual networks. The residual learning mechanism of deep residual networks can effectively alleviate the gradient vanishing problem, making the training of deep networks more stable. In addition, deep residual networks can provide fine feature representations, which helps to generate high-quality segmentation masks and improve the accuracy of the trained image processing models.

[0075] Optionally, after training the image processing model based on sample images from each source domain and at least one sample image from the target domain, if the trained image processing model does not meet the training requirements (e.g., the number of iterations reaches a preset threshold, such as 100 times, and the accuracy reaches a preset accuracy threshold, such as 0.95), multiple new sample images from the source domain and at least one new sample image from the target domain can be obtained to continue training the trained image processing model, or the original sample images from each source domain and at least one sample image from the target domain can be used to continue training the trained image processing model until the trained image processing model meets the training requirements.

[0076] In this embodiment, after obtaining the source domain sample image corresponding to the source domain, a pre-trained color cue is directly used to transform at least one source domain sample image to obtain a target domain sample image. Since the color cue can convert the source domain sample image into an image whose color distribution conforms to the target domain—that is, the color distribution of the obtained target domain sample image conforms to the color distribution characteristics of the target domain—and since the target domain is a different domain from the source domain, training the image processing model using both source and target domain sample images enables the trained image processing model to have better domain generalization ability, thereby improving the accuracy of the image processing model and thus improving the image processing effect. Furthermore, there is no need to separately obtain and annotate the target domain image, effectively reducing the manpower and annotation costs required to collect target domain sample images, which is beneficial for practical applications.

[0077] In some embodiments, the color cue includes a resampling network, a parameter prediction network, and a transformation network. Optionally, the resampling network can be constructed based on a lightweight convolutional neural network, an encoder-decoder structure, or a conditional neural network; the parameter prediction network can be constructed based on a neural network with an attention mechanism such as self-attention or cross-attention, or it can be constructed based on a U-net structure or a deep scene parsing network; the transformation network can be constructed based on a linear network or a deep learning network. It should be noted that the network structure of each network in the color cue can be set according to the actual application scenario, and this application embodiment does not impose specific limitations on this.

[0078] Correspondingly, step S102 above includes:

[0079] For any of the above source domain sample images to be processed:

[0080] The source domain sample image is transformed by the resampling network described above to obtain a resampled image. The transformation process is used to convert the source domain sample image from the initial color space to the target color space.

[0081] The conversion parameters are obtained by analyzing the color values ​​of each pixel in the resampled image using the above parameter prediction network.

[0082] The target domain sample image is determined based on the resampled image and the transformation parameters described above.

[0083] It is understood that the initial color space and the target color space are different color spaces. Since some specific color spaces can better separate the brightness and hue information of colors, they are helpful for processing images with similar or complex hues. Moreover, choosing a suitable color space can reduce the required dimensions of analysis, i.e., reduce computational complexity. Therefore, in this embodiment, the source domain sample image is first converted from the initial color space to the target color space through a resampling network. Subsequent analysis and conversion are performed based on the resampled image in the target color space, which helps to improve the accuracy of the obtained target domain sample image.

[0084] Optionally, the above-mentioned conversion parameters are determined based on the difference between the color distribution of the resampled image (i.e., the source domain sample image) and the color distribution of the target domain. Optionally, the above-mentioned conversion parameters can be determined based on color histogram analysis or color principal component analysis, etc., based on the color values ​​of each pixel in the resampled image. The specific analysis method for color values ​​is not limited in the embodiments of this application.

[0085] After obtaining the source domain sample image (i.e., the resampled image) in the target color space, the parameter prediction network analyzes the color values ​​of each pixel in the resampled image. Since the parameter prediction network in the pre-trained color cue learns the color distribution of the target domain during its training, and the color values ​​of each pigment in the resampled image reflect this color distribution, the parameter prediction network can effectively analyze the difference between the color distribution of the resampled image and the color distribution of the target domain. Based on this difference, it can better determine the transformation parameters used to convert the resampled image into an image that conforms to the color distribution of the target domain. Finally, the transformation network uses these transformation parameters to convert the resampled image into an image whose color distribution conforms to the color distribution characteristics of the target domain, resulting in a highly accurate target domain sample image.

[0086] In some embodiments, the color space of the source domain sample image is a color space where the color channels are not independent, such as YUV (Luma, Blue-difference, Red-difference). In some embodiments, the source domain sample image is an RGB (Red, Green, Blue) image, that is, the initial color space is the RGB color space, and the target color space can be a color space where the color channels are independent, such as HSL (Hue, Saturation, Lightness) or Lab (CIE Lab).

[0087] Since the color channels of an image are independent of each other, the model can perform specific color feature extraction, enhancement, or weakening processing. Therefore, in this embodiment, the source domain sample image is first converted from the initial color space to the target color space to obtain the resampled image corresponding to the target color space. Subsequent analysis of the resampled image then improves the accuracy of the obtained conversion parameters. Furthermore, since adjusting a certain color component does not affect other color components when processing the resampled image based on the conversion parameters, the authenticity and accuracy of the colors can be maintained, thereby improving the quality and accuracy of the obtained target domain sample image.

[0088] It is important to note that when the color channels of the resampled image are independent of each other, the parameter prediction network can determine the transformation parameters for the color values ​​of each pixel in the resampled image by analyzing the color value of each color channel in each pixel separately to obtain the transformation parameters for that color channel. Then, the transformation network processes the color values ​​of each pixel in the resampled image according to the transformation parameters for each color channel to obtain the desired target domain sample image.

[0089] In some embodiments, the color space of the target domain sample image is the initial color space. Optionally, the color value of each pixel in the target color space after conversion can be determined according to the conversion parameters, and then the color value of each pixel in the initial color space can be determined according to the color value of each pixel in the target color space, thereby determining the target domain sample image in the initial color space. In other embodiments, when predicting the conversion parameters based on the resampled image, the prediction can be performed by combining the conversion relationship between the target color space and the initial color space. In this case, according to the predicted conversion parameters, the conversion network can convert the resampled image from the target color space to the initial color space, and its color distribution conforms to the color distribution of the target domain. That is, the resampled image converted according to the conversion parameters is the target domain sample image.

[0090] In this embodiment, the color cue first converts the source domain sample image to a target color space that is conducive to subsequent color distribution analysis and other processing through a resampling network. This allows the parameter prediction network to better analyze the difference between the color distribution of the resampling image and the color distribution of the target domain when analyzing the resampling image, thereby determining the conversion parameters with higher accuracy and improving the accuracy of the obtained target domain sample image.

[0091] It is understood that the color cue mainly achieves the transformation of the source domain sample image through the following steps: the source domain sample image is transformed to obtain a resampled image, then the color value of each pixel in the resampled image is analyzed to obtain the transformation parameters, and then the target domain sample image is determined based on the resampled image and the transformation parameters. That is, in some other embodiments, the structure of the color cue is not limited to the resampling network, parameter prediction network and transformation network, and the above steps can also be implemented through other networks, which will not be elaborated here.

[0092] In some embodiments, the transformation parameters include weights and biases for the mean, and weights and biases for the variance. Determining the target domain sample image based on the resampled image and the transformation parameters includes:

[0093] The first mean and the first variance are determined based on the color values ​​of each pixel in the resampled image.

[0094] The predicted mean is determined based on the first mean, the weights of the means, and the bias.

[0095] The prediction variance is determined based on the first variance, the weights of the variances, and the bias.

[0096] The target domain sample image is determined based on the resampled image, the predicted mean, and the predicted variance.

[0097] Since the mean value of each pixel in an image can well reflect the overall tone of the image, and the variance of each pixel measures the degree of deviation between the pixel's color value and the mean value, that is, reflects the color distribution of each pixel in the image, in this embodiment of the application, the difference between the color distribution of the source domain sample image and the color distribution of the target domain can be measured based on the mean and variance of the color values. That is, it is equivalent to analyzing the difference between the color distribution characteristics such as the uniformity and color richness of the color distribution of the source domain sample image and the corresponding color distribution characteristics of the target domain. Thus, the source domain sample image can be better converted into an image whose color distribution conforms to the color distribution characteristics of the target domain by quantifying the color distribution characteristics and their differences.

[0098] In this process, the mean of the color values ​​can be calculated based on the color values ​​of each pixel in the resampled image to obtain the first mean. Then, based on the first mean and the color values ​​of each pixel, the variance of the color values ​​of each pixel can be calculated to determine the first variance.

[0099] It should be noted that a resampled image can correspond to one first variance, or it can correspond to M first variances (M is the number of pixels in the resampled image, and the variance of the color value of each pixel is its corresponding second variance). In this embodiment, an example of one resampled image corresponding to one first variance is used for illustration.

[0100] In this case, when a resampled image corresponds to a first variance, after calculating the variance of the color values ​​of each pixel, the first variance can be determined based on the mean, median, or mode of the variances corresponding to each pixel. In some embodiments, the median of each variance can be used as the first variance corresponding to the resampled image to reduce the impact of outliers in the variances of each pixel on the transformation parameters.

[0101] After obtaining the first mean and first variance, the first mean can be weighted according to the weights of the mean in the transformation parameters, and then adjusted by the bias of the mean to make the obtained predicted mean better fit the color distribution of the target domain. Similarly, when determining the predicted variance based on the first variance, the first variance is first weighted according to the weights of the variance in the transformation parameters, and then the weighted first variance is adjusted by the bias of the mean to obtain a more accurate predicted variance. It can be understood that the obtained predicted mean is the mean of the color values ​​of each pixel in the predicted source domain sample image when it conforms to the color distribution of the target domain, and the predicted variance is the variance of each pixel in the predicted source domain sample image when it conforms to the color distribution of the target domain.

[0102] In some embodiments, the above-mentioned predicted mean can be expressed in the following form:

[0103] mean trans =mean resample *mean w +mean b

[0104] mean ttrans The mean represents the predicted mean. resample The mean represents the first mean of the resampled image. w The weights representing the mean, mean b This indicates the bias of the mean.

[0105] When a resampled image corresponds to a first variance, the above prediction variance can be expressed in the following form:

[0106] std trans =std resample *std w +std b

[0107] std trans std represents the prediction variance. resample std represents the first variance of the resampled image. w The weights representing variance, std b This indicates the bias of the variance.

[0108] When each resampled pixel corresponds to a first variance, the above prediction variance can be expressed in the following form:

[0109] std trans，i =std resample，i *std w +std b

[0110] std trans，i std represents the prediction variance of the i-th pixel in the resampled image. resample，i std represents the first variance corresponding to the i-th pixel in the resampled image. w The weights representing variance, std b This indicates the bias of the variance.

[0111] like Figure 2 As shown, after obtaining the predicted mean and predicted variance, the resampled image can be transformed based on the predicted mean and predicted variance to obtain the desired target domain sample image.

[0112] In some embodiments, since the target color space is a color space where color channels are independent of each other, in order to improve the accuracy of the obtained target domain sample image, each step from calculating the first mean and the first variance to processing the target domain sample image can be based on color channels. That is, for each color channel, the first mean and the first variance corresponding to that color channel are calculated, and the predicted mean and the predicted variance corresponding to that color channel are determined. Then, based on the predicted mean and the predicted variance corresponding to the color channel, the values ​​of each color channel of each pixel in the resampled image are processed to obtain the required target domain sample image.

[0113] Optionally, when transforming the resampled image based on the predicted mean and the predicted variance, the resampled image can be processed by histogram matching or mapping algorithms to match the mean of the color values ​​of each pixel in the processed resampled image (i.e., the target domain sample image) with the predicted mean, and to match the variance of each pixel in the processed resampled image with the predicted variance.

[0114] For example, a mapping function can be set up to adjust the color values ​​of each pixel in the resampled image based on the predicted mean and the predicted variance, so that the mean color value of each pixel in the adjusted resampled image is close to the predicted mean, and its variance is close to the predicted variance. During the conversion process, the mapping function receives the color values ​​of each pixel in the resampled image, and converts the color values ​​of each pixel into new color values ​​based on the predicted mean and the predicted variance. After obtaining the new color values ​​of each pixel, the image is reconstructed based on these new color values ​​to obtain the desired target domain sample image.

[0115] In this embodiment, the parameter prediction network combines the color distribution features of the resampled image and the color distribution features of the target domain to predict the weights and biases of the mean and variance of the color values ​​in the resampled image. Then, based on the weights and biases, the mean and variance of the color values ​​of each pixel after the resampled image is transformed into the target domain can be analyzed, i.e., the predicted mean and predicted variance are obtained. Since the predicted mean can better reflect the overall hue of the target domain sample image to be obtained, and the predicted variance reflects the color distribution of each pixel in the target domain sample image to be obtained, the resampled image is transformed based on the predicted mean and predicted variance, so that the mean and variance of the actually obtained target domain sample image are as close as possible to the predicted mean and predicted variance, thereby improving the accuracy of the actually obtained target domain sample image.

[0116] In some embodiments, the number of resampled images is greater than 1. Before analyzing the color values ​​of each pixel in the resampled images using the parameter prediction network to obtain the conversion parameters, the method further includes:

[0117] For any of the above-mentioned resampled images, calculate the mean and variance of the color values ​​of each pixel in the resampled image to obtain the corresponding second mean and the corresponding second variance.

[0118] Calculate the mean and variance of each of the above second means to obtain the third mean and the corresponding third variance.

[0119] Calculate the mean and variance of each of the above second variances to obtain the fourth mean and the corresponding fourth variance.

[0120] The resampled images are normalized based on the second mean, the second variance, the third mean, the third variance, the second variance, the fourth mean, and the fourth variance to obtain the normalized resampled images.

[0121] Correspondingly, the color values ​​of each pixel in the resampled image are analyzed by the parameter prediction network to obtain the transformation parameters, including:

[0122] The above parameters are used to predict the color values ​​of each pixel in the normalized resampled image by analyzing the color values ​​of each pixel, and the above conversion parameters are obtained.

[0123] In this embodiment, multiple source domain sample images are processed by a color cue. After the resampling network transforms each source domain sample image, multiple resampling images are obtained. To further improve the accuracy of the obtained target domain sample images, each resampling image can be normalized to unify the color value range of each resampling image. This also helps to enhance the contrast of each resampling image, making the color distribution characteristics of the resampling images more obvious, which is beneficial for the prediction of subsequent transformation parameters.

[0124] Before normalizing each resampled image, the mean and variance of the mean can be calculated based on the mean and variance of each image.

[0125] Specifically, for any resampled image, the mean of the color values ​​of each pixel in the resampled image is calculated to obtain the second mean corresponding to the resampled image. Based on the second mean, the variance of the color values ​​of each pixel is calculated, and then the second variance corresponding to the resampled image is determined based on the variance of the color values ​​of each pixel. For example, the median of the variances of the color values ​​of each pixel can be used as the second variance, or the mean of the middle N (N is greater than 1 and less than M, where M is the number of variances of color values) variances after sorting can be used as the second variance. It can be understood that in the embodiments of this application, one resampled image corresponds to one second mean and one second variance.

[0126] like Figure 3 As shown, after obtaining the second mean and the corresponding second variance for each resampled image, the mean of each second mean is calculated to obtain the corresponding third mean, and based on the third mean, the variance of each second mean is calculated to obtain the third variance of each second mean.

[0127] Similarly, for each second variance, calculate the mean of each second variance to obtain the corresponding fourth mean, and based on the fourth mean, calculate the variance corresponding to each second variance to obtain the fourth variance corresponding to each second variance.

[0128] For example, suppose we have three resampled images: image A, image B, and image C, each containing 300 pixels. During normalization, based on the color values ​​of these 300 pixels, we can calculate the second mean and second variance for images A, B, and C respectively, obtaining the second mean (meanA) and second variance (stdA) for image A, the second mean (meanB) and second variance (stdB) for image B, and the second mean (meanC) and second variance (stdC) for image C.

[0129] Then, based on the three second means (meanA, meanB, and meanC), the third mean meanM3 is calculated. Then, based on the third mean meanM3 and the three second means, the third differences corresponding to each second mean are calculated: std31, std32, and std33.

[0130] Similarly, based on the three second variances (stdA, stdB, stdC), the fourth mean M4 is calculated. Then, based on the fourth mean M4 and the three second variances, the fourth variances corresponding to each second variance are calculated: std41, std42, and std43.

[0131] After obtaining the mean and variance, the resampled images can be normalized based on the second mean, second variance, third mean, third variance, fourth mean, and fourth variance to obtain normalized resampled images.

[0132] It should be noted that in the embodiments of this application, X (X is greater than 1) resampled images correspond to a third mean, X third variances, a fourth mean, and X fourth variances.

[0133] As an example, when normalizing each resampled image, the personalized mean of each resampled image can be dynamically set based on the second mean, third mean, and third difference of each resampled image.

[0134] Furthermore, the variance scaling factor for each resampled image can be dynamically set based on the second variance, fourth mean, and each fourth variance of each resampled image.

[0135] Finally, for any resampled image, the color values ​​of each pixel in the resampled image are transformed according to the personalized base value and variance scaling factor corresponding to the resampled image, so as to obtain the normalized resampled image.

[0136] As an example, the color value of any pixel after transformation can be represented in the following form:

[0137]

[0138] Among them, Z yi Let x represent the color value of the i-th pixel in the resampled image y after transformation. yi The mean value represents the color value of the i-th pixel in the resampled image y before transformation. ya y represents the personalized mean of the resampled image y, mean3 represents the third mean, and b represents the variance scaling factor of the resampled image y.

[0139] As another example, when normalizing each resampled image, a target third difference (such as the mean of each third difference) can be determined based on each third difference. Then, the second mean corresponding to each resampled image can be adjusted according to the target third difference to obtain the adjusted second mean. Similarly, a target fourth difference (such as the median of each fourth difference) can be determined based on each fourth difference. Then, the second difference corresponding to each resampled image can be adjusted according to the target fourth difference to obtain the adjusted second difference. Finally, for any resampled image, the color value of each pixel in the resampled image can be transformed according to the adjusted second mean and the adjusted second difference to obtain the normalized resampled image.

[0140] In this embodiment, during normalization, the resampled image is simultaneously normalized based on each second mean, third mean, third difference, second variance, fourth mean, and fourth variance. Since the second mean and second variance are calculated based on the color values ​​of each pixel in the resampled image, reflecting the overall hue and color distribution of the resampled image, while the third mean and third difference reflect the overall hue of each resampled image and the differences in overall hue between resampled images, and the fourth mean and fourth variance reflect the overall color distribution of each resampled image and the differences in color distribution between resampled images, normalizing each resampled image in the above manner can unify the color value range of each resampled image, reduce the differences in color distribution characteristics between resampled images, and simultaneously enhance the contrast of each resampled image, making the color distribution characteristics of the resampled image more obvious, which is beneficial to improving the accuracy of the subsequent predicted transformation parameters.

[0141] In some embodiments, the color cue is trained through the following steps:

[0142] A1. Obtain the target domain image corresponding to the above target domain.

[0143] A2. The target domain image is transformed using the above-mentioned resampling network to obtain the above-mentioned resampled image.

[0144] A3. The color values ​​of each pixel in the above resampled image are analyzed by the above parameter prediction network to obtain the above conversion parameters.

[0145] A4. Determine the predicted image based on the above resampled image and the above transformation parameters. The predicted image is the image in the above initial color space.

[0146] A5. Update the parameters of the above parameter prediction network based on the difference between the target domain image and the above predicted image to obtain the updated parameter prediction network.

[0147] A6. Based on the updated parameters, predict the network to determine the pre-trained color cue.

[0148] The aforementioned target domain image is an image that conforms to the target domain. In this embodiment of the application, the target domain image can be any image that conforms to the target domain. The target domain image can be an unlabeled image, so that when training the color cue, there is no need to label the target domain image, which can effectively reduce the training cost.

[0149] In some embodiments, the target domain image can be an image that conforms to the target domain, determined according to the actual application scenario of the image processing model. For example, when the actual application scenario of the image processing model is pedestrian re-identification, the target domain image can be a pedestrian image that conforms to the target domain, so that the color cue can learn the color distribution of the pedestrian image in the target domain, which is beneficial to the subsequent conversion processing of the source domain sample image.

[0150] After obtaining the target domain image corresponding to the target domain, the target domain image is first converted from the initial color space to the target color space through a resampling network. Then, the parameter prediction network analyzes the color distribution of the resampled image based on the color value of each pixel in the resampled image, predicts the conversion parameters, and processes the resampled image according to the conversion parameters to obtain the predicted image in the initial color space.

[0151] After obtaining the predicted image in the initial color space, the parameters of the parameter prediction network can be updated directly based on the difference between the target domain image and the predicted image. Specifically, a loss value can be calculated based on a preset loss function, the target domain image, and the predicted image, and the parameters of the parameter prediction network can be updated based on this loss value to obtain the updated parameter prediction network.

[0152] In some embodiments, the above loss value can be expressed in the following form:

[0153]

[0154] Where N is the number of target domain images participating in the training in the current round, mean ori The mean is the average color value of each pixel in the target domain image. trans To predict the mean color value of each pixel in an image, std ori The target variance (e.g., the mean of all variances) is determined by the variance of the color values ​​of each pixel in the target domain image. trans The target variance is determined to predict the variance corresponding to the color value of each pixel in the image.

[0155] Optionally, after obtaining the updated parameter prediction network, if the updated parameter prediction network does not meet the training requirements (e.g., the number of iterations is less than the threshold of 100), steps A1 to A5 can be repeated, or steps A2 to A5 can be repeated, until the latest updated parameter prediction network meets the training requirements, and the pre-trained color cue is determined based on the latest updated parameter prediction network.

[0156] For example, after obtaining the updated parameter prediction network, steps A2 to A4 above can be performed based on one or more target domain images. In step A3, the color values ​​of each pixel in the resampled image are analyzed by the updated parameter prediction network to obtain the transformation parameters. Then, step A4 is performed to obtain the corresponding prediction image. At this time, it can be analyzed whether the difference between the prediction image and the target domain image is less than or equal to the difference threshold (e.g., 5%).

[0157] If the difference between the predicted image and the target domain image is less than or equal to the difference threshold, the updated parameter prediction network can be considered to meet the training requirements. In this case, the pre-trained color cue can be determined directly using the updated parameter prediction network. If the difference between the predicted image and the target domain image is greater than the difference threshold, the updated parameter prediction network can be considered not to meet the training requirements. In this case, step A5 can be executed to update the updated parameter prediction network based on the difference between the prediction network and the target domain image. Then, steps A2 to A4 are executed based on the latest parameter prediction network, including the steps of difference judgment and determining whether to continue updating the parameter prediction network based on the difference result, until the latest updated parameter prediction network meets the training requirements.

[0158] In this embodiment, the color cue is trained using a target domain image corresponding to the target domain. This target domain image is an unlabeled image that conforms to the target domain, eliminating the need for annotation and other processing, thus reducing the training cost of the color cue. During training, the parameter prediction network is updated based on the difference between the predicted image and the target domain image. This allows the parameter prediction network to effectively learn the color distribution features of the target domain image during iteration, reducing the difference between the predicted and target domain images, i.e., reducing the difference in color distribution between the predicted and target domain images, resulting in a highly accurate pre-trained color cue.

[0159] In some embodiments, the above-described processing of at least one source domain sample image using a pre-trained color cue to obtain a target domain sample image includes:

[0160] At least one source domain sample image is processed by multiple color cues to obtain a corresponding target domain sample image, wherein different color cues correspond to different target domains.

[0161] To further improve the generalization ability of the image processing model, multiple color cues corresponding to target domains can be pre-trained. Different color cues correspond to different target domains. It can be assumed that the target domain sample images converted by different color cues have different color distributions.

[0162] After obtaining multiple source domain sample images corresponding to the source domain, at least two color cues can be selected from multiple pre-trained color cues for data augmentation. That is, at least two color cues are used to process at least one source domain sample image to obtain the corresponding target domain sample image, thus obtaining target domain sample images corresponding to different target domains.

[0163] In some embodiments, the data augmentation is not limited to which target domains; that is, at least two color cues can be randomly selected from a plurality of color cues as target color cues, and data augmentation can be performed using the selected target color cues.

[0164] For example, suppose an image processing model is applied to a face recognition task, and suppose the source domain is a camera of model A. The source domain sample image can be a face image captured by a camera of model A. Suppose there are the following 5 target domains: cameras of model B, model C, model D, model E, and model F, and their corresponding color cues: color c, color cue d, color cue e, and color cue f. Color c, color cue e, and color cue f are randomly selected as target color cues.

[0165] During data augmentation, five source domain sample images can be randomly selected from multiple source domain sample images. These five source domain sample images are then transformed using three target color cues to obtain target domain sample images corresponding to three target domains: five sample images corresponding to camera model C, five sample images corresponding to camera model E, and five sample images corresponding to camera model F. Through this processing, images from multiple target domains different from the source domains are obtained as sample images for the image processing model. This allows the image processing model to learn the features of multiple target domains different from the source domains better during training, thereby improving the generalization ability of the image processing model from the source domains and various target domains to other domains.

[0166] In this embodiment, multiple color cues are used to transform the source domain sample images to obtain multiple target domain sample images. This enables the image processing model to learn features of multiple domains different from the source domain when it is trained based on the source domain sample images and these target domain sample images. As a result, it can better generalize the knowledge learned from the source domain and each target domain to other domains, further improving its domain generalization ability and thus improving the image processing effect.

[0167] Figure 4 A schematic flowchart of the image processing method provided in an embodiment of this application is shown, and is described in detail below:

[0168] Step S401: Obtain the image to be processed.

[0169] Step S402: Input the image to be processed into the pre-trained image processing model to obtain the image processing result output by the image processing model. The image processing model is trained according to the model training method described in the above embodiment.

[0170] The method for acquiring the image to be processed can be set according to the actual application scenario. This application embodiment does not impose specific restrictions on the method for acquiring the image to be processed.

[0171] For example, when applying the above image processing model to pedestrian tracking tasks in autonomous driving scenarios, the image to be processed can be an image captured by the vehicle's camera equipment (such as a forward-facing camera and a reversing camera); when applying the above image processing method to defect detection tasks of assembly line products, the image to be processed can be an image captured by monitoring cameras installed on each assembly line.

[0172] In this embodiment, since the target domain sample image corresponding to the target domain is also used for training during the training process of the image processing model, and the target domain and the source domain are different domains, the image processing model can learn the features of other domains during the training process, rather than being limited to the features of the source domain. This makes the trained image processing model have better domain generalization ability, and can better process the images to be processed that do not belong to the source domain, thereby improving the accuracy of the obtained image processing results, that is, improving the image processing effect.

[0173] The following describes the process of the model training method and image processing method provided in the embodiments of this application, in conjunction with the target recognition application scenario.

[0174] In this embodiment, the various domains can be determined based on the different camera models. Assume the source domain is camera model A, and there are three target domains: camera model B, camera model C, and camera model D. In this case, the image captured by camera model B can be used as its corresponding training image (i.e., the target domain image, referred to as training image B). Training image B is used to train a color cue, resulting in color cue B. Similarly, for camera model C and camera model D, the images captured by them are also used as corresponding training images to train the color cue, resulting in color cue C and color cue D. Further details are omitted here.

[0175] The training images for each color cue can be images containing various targets (such as pedestrians, vehicles, and various animals), enabling the corresponding color cue to learn the color distribution features of various targets in the corresponding target domain.

[0176] Suppose that 100 source domain sample images corresponding to the source domain (camera of model A) are obtained. The source domain sample images are images containing one or more targets with category labels and location labels for each target.

[0177] Twenty source domain sample images are randomly selected, and color cue C and color cue D are randomly selected to process these 20 source domain sample images, resulting in 20 target domain sample images corresponding to camera model C and camera model D. At this point, each target domain sample image is an image whose color distribution conforms to the color distribution of the target domain, and each target domain sample image contains one or more targets with category and location labels for each target.

[0178] At this point, a total of 140 sample images can be obtained (including 100 source domain sample images corresponding to camera model A, 20 target domain sample images corresponding to camera model C, and 20 target domain sample images corresponding to camera model D). These 140 sample images are used to train the image processing model to obtain the trained image processing model.

[0179] If the trained image processing model does not meet the training requirements (e.g., accuracy of 0.98), a new source domain sample image can be obtained. Based on the new source domain sample image, the following steps are repeated: converting the source domain sample image using a color cue to obtain the target domain sample image, and training the image processing model using each sample image, until the latest trained image processing model meets the training requirements, thus obtaining the pre-trained image processing model.

[0180] At this point, the pre-trained image processing model can be used for target recognition.

[0181] After acquiring the image to be processed through the camera, the image to be processed is input into a pre-trained image processing model to obtain the image processing result output by the image processing model. The image processing result can indicate the category and location of each target in the image to be processed.

[0182] Furthermore, assuming the above image processing model is applied to pedestrian re-identification in target recognition, in order to improve the accuracy of pedestrian re-identification, the source domain sample image can be an image containing human bodies captured by a camera of model A, and the source domain sample image carries the identity tags of each human body; correspondingly, the target domain sample image obtained by the color cue device through the transformation processing of the source domain sample image is an image containing human bodies that conforms to the color distribution of the target domain, and carries the identity tags corresponding to the human bodies.

[0183] During the training of the UI image processing model based on source domain sample images and target domain sample images, the loss value can be calculated based on the loss function of cross-entropy loss and triplet loss, and the parameters of the image processing model can be updated based on the loss value.

[0184] As an example, the above loss value can be expressed in the following form:

[0185]

[0186] Where Loss is the aforementioned loss value, y is the label value of the human target in the sample image, and p i To predict the probability that a target in a sample image is a person with a certain identity, d p d n α and α are parameters in the triplet function, representing the positive pair distance, negative pair distance, and margin (i.e., the minimum difference threshold between the positive and negative pair distances), respectively. o -d n +α] + Indicates (d) p -d n +α, 0).

[0187] After training, the image processing model can be used for pedestrian re-identification. Correspondingly, its image processing results can indicate the identity of the human body present in the image.

[0188] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0189] Example 2:

[0190] Corresponding to the model training method described in the above embodiments, Figure 5 A structural block diagram of the model training apparatus provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0191] Reference Figure 5 The device includes: a source domain sample acquisition module 51, a target domain sample acquisition module 52, and a training module 53. Among them,

[0192] The source domain sample acquisition module 51 is used to acquire multiple source domain sample images corresponding to the source domain.

[0193] The target domain sample acquisition module 52 is used to process at least one of the source domain sample images through a pre-trained color cue to obtain a corresponding target domain sample image. The color cue is used to convert the source domain sample image into an image whose color distribution conforms to the target domain to obtain the target domain sample image. The target domain is different from the source domain.

[0194] Training module 53 is used to train the image processing model based on each of the above-mentioned source domain sample images and at least one of the above-mentioned target domain sample images to obtain the pre-trained image processing model.

[0195] In this embodiment, after obtaining the source domain sample image corresponding to the source domain, a pre-trained color cue is directly used to transform at least one source domain sample image to obtain a target domain sample image. Since the color cue can convert the source domain sample image into an image whose color distribution conforms to the target domain—that is, the color distribution of the obtained target domain sample image conforms to the color distribution characteristics of the target domain—and since the target domain is a different domain from the source domain, training the image processing model using both source and target domain sample images enables the trained image processing model to have better domain generalization ability, thereby improving the accuracy of the image processing model and thus improving the image processing effect. Furthermore, there is no need to separately obtain and annotate the target domain image, effectively reducing the manpower and annotation costs required to collect target domain sample images, which is beneficial for practical applications.

[0196] In some embodiments, the color cue includes a resampling network, a parameter prediction network, and a transformation network, and the target domain sample image acquisition module 52 includes:

[0197] The conversion unit is used to perform conversion processing on the source domain sample image through the resampling network to obtain a resampled image. The conversion processing is used to convert the source domain sample image from the initial color space to the target color space.

[0198] The analysis unit is used to analyze the color values ​​of each pixel in the resampled image through the above parameter prediction network to obtain the transformation parameters.

[0199] The target domain sample image determination unit is used to determine the target domain sample image based on the resampled image and the transformation parameters.

[0200] In some embodiments, the transformation parameters include weights and biases for the mean, and weights and biases for the variance. The target domain sample image acquisition module 52 further includes:

[0201] The first calculation unit is used to determine the first mean and the first variance based on the color values ​​of each pixel in the resampled image.

[0202] The predictive mean determination unit is used to determine the predictive mean based on the first mean, the weight of the mean, and the bias.

[0203] The prediction variance determination unit is used to determine the prediction variance based on the first variance, the weights of the variances, and the bias.

[0204] The target domain sample image determination unit is used to determine the target domain sample image based on the resampled image, the predicted mean, and the predicted variance.

[0205] In some embodiments, the number of resampled images is greater than 1, and the model training apparatus further includes:

[0206] The second calculation module is used to calculate the mean and variance of the color values ​​of each pixel in any of the above-mentioned resampled images, so as to obtain the second mean and the corresponding second variance of the above-mentioned resampled image.

[0207] The third calculation module is used to calculate the mean and variance of each of the above-mentioned second means, and to obtain the third mean and the corresponding third variance.

[0208] The fourth calculation module is used to calculate the mean and variance of each of the above-mentioned second variances, so as to obtain the fourth mean and the corresponding fourth variance.

[0209] The normalization module is used to normalize each of the above-mentioned resampled images based on each of the above-mentioned second means, each of the above-mentioned second variances, the above-mentioned third means, each of the above-mentioned third variances, each of the above-mentioned second variances, the above-mentioned fourth means, and each of the above-mentioned fourth variances, to obtain each of the above-mentioned normalized resampled images.

[0210] Correspondingly, the analysis unit is specifically used for:

[0211] The above parameters are used to predict the color values ​​of each pixel in the normalized resampled image by analyzing the color values ​​of each pixel, and the above conversion parameters are obtained.

[0212] In some embodiments, the above-described model training apparatus further includes:

[0213] The target domain image acquisition module is used to acquire the target domain image corresponding to the above target domain.

[0214] The conversion module is used to convert the target domain image through the resampling network to obtain the resampled image.

[0215] The analysis module is used to analyze the color values ​​of each pixel in the above-mentioned resampled image and the prediction network based on the above-mentioned parameters to obtain the above-mentioned transformation parameters.

[0216] The prediction module is used to determine a prediction image based on the resampled image and the transformation parameters, wherein the prediction image is an image in the initial color space.

[0217] The update module is used to update the parameters of the parameter prediction network based on the difference between the target domain image and the predicted image, so as to obtain the updated parameter prediction network.

[0218] The color cue acquisition module is used to determine the pre-trained color cue based on the updated parameters of the above-mentioned prediction network.

[0219] In some embodiments, the target domain sample image acquisition module 52 includes:

[0220] The multi-target domain conversion unit is used to process at least one source domain sample image through multiple color cues to obtain a corresponding target domain sample image, wherein different color cues correspond to different target domains.

[0221] Figure 6 A structural block diagram of an image processing apparatus provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0222] Reference Figure 6 The device includes: an image acquisition module 61 and a processing module 62. Among them,

[0223] The image acquisition module 61 is used to acquire the image to be processed.

[0224] The processing module 62 is used to input the image to be processed into a pre-trained image processing model to obtain the image processing result output by the image processing model. The image processing model is trained according to the model training method described in the above embodiment.

[0225] In this embodiment, since the target domain sample image corresponding to the target domain is also used for training during the training process of the image processing model, and the target domain and the source domain are different domains, the image processing model can learn the features of other domains during the training process, rather than being limited to the features of the source domain. This makes the trained image processing model have better domain generalization ability, and can better process the images to be processed that do not belong to the source domain, thereby improving the accuracy of the obtained image processing results, that is, improving the image processing effect.

[0226] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0227] Example 3:

[0228] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 7 of this embodiment includes: at least one processor 70 ( Figure 7 The diagram shows only one processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, which, when executed, performs the steps in any of the above-described method embodiments.

[0229] The electronic device 7 can be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of electronic device 7 and does not constitute a limitation on electronic device 7. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0230] The processor 70 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0231] In some embodiments, the memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. In other embodiments, the memory 71 may be an external storage device of the electronic device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 7. Furthermore, the memory 71 may include both internal and external storage units of the electronic device 7. The memory 71 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0232] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0233] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0234] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0235] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0236] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0237] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0238] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0239] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0240] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0241] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A model training method, characterized in that, include: Obtain multiple source domain sample images corresponding to the source domain; At least one source domain sample image is processed by a pre-trained color cue to obtain a corresponding target domain sample image. The color cue is used to convert the source domain sample image into an image whose color distribution conforms to the target domain to obtain the target domain sample image. The target domain is different from the source domain. The image processing model is trained based on each of the source domain sample images and at least one of the target domain sample images to obtain the pre-trained image processing model.

2. The model training method as described in claim 1, characterized in that, The color cue includes a resampling network, a parameter prediction network, and a transformation network. The process of processing at least one source domain sample image using the pre-trained color cue to obtain a corresponding target domain sample image includes: For any one of the source domain sample images to be processed: The source domain sample image is transformed by the resampling network to obtain a resampled image. The transformation process is used to convert the source domain sample image from the initial color space to the target color space. The parameter prediction network is used to analyze the color values ​​of each pixel in the resampled image to obtain the conversion parameters. The target domain sample image is determined based on the resampled image and the transformation parameters.

3. The model training method as described in claim 2, characterized in that, The transformation parameters include weights and biases for the mean, and weights and biases for the variance. Determining the target domain sample image based on the resampled image and the transformation parameters includes: The first mean and the first variance are determined based on the color values ​​of each pixel in the resampled image; The predicted mean is determined based on the first mean, the weight of the mean, and the bias. The prediction variance is determined based on the first variance, the weights of the variances, and the bias. The target domain sample image is determined based on the resampled image, the predicted mean, and the predicted variance.

4. The model training method as described in claim 2, characterized in that, The number of resampled images is greater than 1. Before analyzing the color values ​​of each pixel in the resampled images through the parameter prediction network to obtain the transformation parameters, the method further includes: For any of the resampled images, calculate the mean and variance of the color values ​​of each pixel in the resampled image to obtain the second mean and the corresponding second variance of the resampled image; Calculate the mean and variance of each of the second means to obtain the third mean and the corresponding third variance; Calculate the mean and variance of each of the second variances to obtain the fourth mean and the corresponding fourth variance; The resampled images are normalized based on each of the second means, each of the second variances, the third means, each of the third variances, each of the second variances, the fourth means, and each of the fourth variances to obtain the normalized resampled images. Correspondingly, the step of analyzing the color values ​​of each pixel in the resampled image through the parameter prediction network to obtain the transformation parameters includes: The conversion parameters are obtained by analyzing the color values ​​of each pixel in the normalized resampled image through the parameter prediction network.

5. The model training method as described in claim 2, characterized in that, The color cue is trained through the following steps: Obtain the target domain image corresponding to the target domain; The target domain image is transformed using the resampling network to obtain the resampled image; The conversion parameters are obtained by analyzing the parameter prediction network and the color values ​​of each pixel in the resampled image. A predicted image is determined based on the resampled image and the transformation parameters, wherein the predicted image is an image in the initial color space; The parameters of the parameter prediction network are updated based on the difference between the target domain image and the predicted image to obtain the updated parameter prediction network. The pre-trained color cue is determined based on the updated parameters of the prediction network.

6. The model training method according to any one of claims 1 to 5, characterized in that, The step of processing at least one source domain sample image using a pre-trained color cue to obtain a corresponding target domain sample image includes: At least one source domain sample image is processed by multiple color cues to obtain a corresponding target domain sample image, wherein different color cues correspond to different target domains.

7. An image processing method, characterized in that, include: Obtain the image to be processed; The image to be processed is input into a pre-trained image processing model to obtain the image processing result output by the image processing model, which is trained according to any one of claims 1 to 6.

8. A model training device, characterized in that, include: The source domain sample acquisition module is used to acquire multiple source domain sample images corresponding to the source domain; The target domain sample acquisition module is used to process at least one of the source domain sample images through a pre-trained color cue to obtain a corresponding target domain sample image. The color cue is used to convert the source domain sample image into an image whose color distribution conforms to the target domain to obtain the target domain sample image. The target domain is different from the source domain. The training module is used to train the image processing model based on each of the source domain sample images and at least one of the target domain sample images to obtain the pre-trained image processing model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6 or the method as described in claim 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6 or the method as described in claim 7.

Images