Glass image color migration system, migration method, device and medium

Abstract

The invention discloses a glass image color migration system, migration method and device and a medium, and belongs to the technical field of image style migration, and the glass image color migration method comprises the following steps: S100, obtaining an actual glass image; and S200, receiving an actual glass image, and performing color migration on the actual glass image by using the glass image color migration model to generate an actual glass color migration image. According to the invention, a large amount of glass image sample data with different color distribution can be obtained, the diversity of the glass image sample data set is expanded, the influence of the color change of the surrounding environment on the detection result of a subsequent glass quality image monitoring model is reduced, and the generalization ability of the glass quality image monitoring model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image style transfer, and in particular relates to a glass image color transfer system, a transfer method, a device, and a medium. Background Art

[0002] In the process of rapid and continuous development of science and technology, the integration of artificial intelligence into all aspects of life and production has become an unstoppable trend and an irresistible trend. Among the many application scenarios of artificial intelligence, the field of image detection is undoubtedly the focus of attention. Image detection based on artificial intelligence has gradually been applied in many fields such as autonomous driving, security monitoring, industrial quality inspection, etc., especially in the glass quality monitoring scenario. The production of defective glass is an unavoidable problem, such as broken edges, cracks, scratches, etc. However, in the actual glass production process, the occurrence of defects and defects is often rare, and it is difficult to collect enough image sample data, resulting in uneven distribution of glass defect sample data. The generated artificial intelligence-based glass quality image monitoring model is difficult to achieve the accuracy required for industrial level.

[0003] Moreover, the glass quality image monitoring model based on artificial intelligence is often affected by external colors. For example, changes in the color of ordinary lights in the surrounding environment after the glass is produced will have a key impact on the accurate detection results of the glass quality image monitoring model. Although the use of image color migration technology can increase the diversity of glass image monitoring data and reduce the impact of changes in the color of surrounding light on glass quality detection results, it can improve the generalization of the glass quality image monitoring model to a certain extent. However, the glass defect images in the glass production process often have a small amount of data and most of the glass image data are basically similar. Too similar glass image data will cause the glass quality image monitoring model to overfit, fall into local optimal solutions, have poor generalization, and data deviation. When the image color migration technology injects the color information of the target glass image, it is difficult to affect the glass image and achieve the color change effect.

[0004] At present, StyleGAN uses AdaIN to inject style transfer information, which can affect the image. However, it cannot accurately adjust the color because it cannot confirm the latent code corresponding to the color feature. HistoGAN inherits the style transfer information injection mode of StyleGAN and uses color histogram as the injection information. The simple information modulation method cannot correctly extract color information, nor can it ensure that the color information has correctly changed the color of the image. The conditional diffusion model has great prospects in the field of style transfer. The diffusion model's requirements for the amount and diversity of data and the training overhead limit the application of the diffusion model in glass quality image monitoring. Text guidance is increasingly widely used in diffusion models, especially in image generation tasks. It can control the content and style of generated images through natural language descriptions. The addition of text guidance enables the diffusion model to better generate images that match the text description, greatly expanding the application scenarios of the generation model. However, text and image are two different modalities. How to effectively establish a connection between the two modalities is a major challenge. Color information is also difficult to simply use text descriptions to enable the model to accurately generate images of corresponding colors. Therefore, when the image sample data is small and similar, how to reduce the impact of changes in the surrounding environment on the image detection results, alleviate the shortage of image sample data in glass defect detection, and improve the generalization of the glass quality image monitoring model are technical problems that need to be solved urgently. Summary of the invention

[0005] The present invention application provides a glass image color migration system, migration method, device, and medium, aiming to partially or completely solve the technical problems in existing glass quality monitoring that the color changes of the surrounding environment have a great impact on the image detection results, the glass defect image sample data is insufficient, and the generalization of the glass quality image monitoring model is insufficient. The present invention application can obtain a large amount of glass image sample data with different color distributions, expand the diversity of the glass image sample data set, reduce the impact of the color changes of the surrounding environment on the detection results of the glass quality image monitoring model, and improve the robustness of the glass quality image monitoring model. In order to achieve the above purpose, the present invention application adopts the following technical solutions:

[0006] In a first aspect, a glass image color migration system comprises:

[0007] Image input module, glass image color migration model, image output module;

[0008] An image input module is used to obtain an actual glass image; a glass image color migration model is used to receive an actual glass image and perform color migration on the actual glass image to generate an actual glass color migration image; an image output module is used to output the actual glass color migration image;

[0009] The glass image color migration model includes: a color modulation module and a generation module; the color modulation module receives a target image, modulates the color information of the target image, and obtains the required injected color control information; the generation module generates an actual glass color migration image based on at least the actual glass image and the required injected color control information.

[0010] Optionally, the color modulation module includes a color histogram unit, a modulation unit and a demodulation unit, the modulation unit includes multiple downsampling subunits, a first convolution subunit one, and a first convolution subunit two, each downsampling subunit includes a convolution subunit one and a batch normalization subunit one; the demodulation unit includes multiple upsampling subunits, a second convolution subunit and an adaptive pooling subunit, each upsampling subunit includes a convolution subunit two and a batch normalization subunit two; the color histogram unit converts the target image into a target color histogram, the modulation unit modulates the target color histogram to obtain the color control information to be injected; the demodulation unit demodulates the color control information to be injected to obtain a demodulated color intuitive map.

[0011] Optionally, the generation module includes: a 1st upsampling unit, a 2nd upsampling unit, ..., a jth upsampling unit, ..., a Pth upsampling unit, the 1st upsampling unit, the 2nd upsampling unit, ..., the jth upsampling unit, ..., the Pth upsampling unit are respectively used to obtain the 1st tensor data, the 2nd tensor data, ..., the jth tensor data, ..., the Pth tensor data, wherein the jth upsampling unit includes: a jth modulation subunit and a jth convolution activation unit, the jth modulation subunit is used to receive and modulate the image style information and the noise information, the jth convolution activation unit is used to perform convolution fusion activation on the modulated image style information and the noise information to obtain the jth tensor data, j is a positive integer, j=1, 2, ..., P;

[0012] The generation module also includes: a color injection unit and a generation unit, wherein the color injection unit generates the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., the Pth glass color migration image, respectively, according to at least the color control information to be injected and the first tensor data, at least the color control information to be injected and the second tensor data, ..., at least the color control information to be injected and the jth tensor data, ..., at least the color control information to be injected and the Pth tensor data, wherein the color injection unit includes a first convolution activation unit, a second convolution activation unit, and a fusion convolution. The first convolution activation unit and the second convolution activation unit are arranged in parallel, the first convolution activation unit includes a plurality of convolution units 1 and a plurality of activation units 1, and the second convolution activation unit includes a plurality of convolution units 2 and a plurality of activation units 2; the first convolution activation unit performs convolution fusion activation at least according to the received color control information required for injection and the jth tensor data to obtain the jth color injection data 1; the second convolution activation unit performs convolution fusion activation at least according to the received color control information required for injection and the jth tensor data to obtain the jth color injection data 2; the fusion convolution unit performs fusion convolution on the jth color injection data 1 and the jth color injection image data 2 to obtain the jth glass color migration image;

[0013] The generating unit fuses the first glass color transition image, the second glass color transition image, ..., the jth glass color transition image, ..., the Pth glass color transition image to generate an actual glass color transition image.

[0014] In a second aspect, a glass image color migration method is provided, using any one of the glass image color migration systems described in the first aspect, comprising:

[0015] Step S100: acquiring an actual glass image;

[0016] Step S200: receiving an actual glass image, and performing color migration on the actual glass image using a glass image color migration model to generate an actual glass color migration image;

[0017] Step S200 includes:

[0018] Step S201: constructing a glass image color migration model;

[0019] Step S202: receiving a target image, modulating the color information of the target image using a glass image color transfer model, obtaining the required injected color control information, and generating an actual glass color transfer image at least based on the actual glass image and the required injected color control information.

[0020] Optionally, the glass image color migration model includes a color modulation module, a generation module and an identification module; step S201 includes:

[0021] Step S2011: training and constructing a color modulation module;

[0022] Step S2012: Based on the color modulation module, a glass image color migration model is trained and constructed.

[0023] Optionally, step S2011 includes:

[0024] Step S20111: obtaining an ImageNet image dataset, where the ImageNet image dataset includes a 1st ImageNet image, a 2nd ImageNet image, ..., an ith ImageNet image, ..., an Nth ImageNet image;

[0025] Step S20112: converting the 1st ImageNet image, the 2nd ImageNet image, ..., the i-th ImageNet image, ..., the N-th ImageNet image into the 1st color histogram, the 2nd color histogram, ..., the i-th color histogram, ..., the N-th color histogram;

[0026] Step S20113: modulating the first color histogram, the second color histogram, ..., the i-th color histogram, ..., and the N-th color histogram respectively to obtain the first color control information, the second color control information, ..., the i-th color control information, ..., and the N-th color control information; demodulating the first color control information, the second color control information, ..., the i-th color control information, ..., and the N-th color control information to obtain the first demodulated color histogram, the second demodulated color histogram, ..., the i-th demodulated color histogram, ..., and the N-th demodulated color histogram;

[0027] Step S20114: calculating the sum of the mean square errors of the first demodulated color histogram, the second demodulated color histogram, ..., the i-th demodulated color histogram, ..., the N-th demodulated color histogram and the first color histogram, the second color histogram, ..., the i-th color histogram, ..., the N-th color histogram respectively as a color histogram loss function;

[0028] Step S20115: Use the optimizer to optimize the color histogram loss function, and repeat steps S20112, S20113, and S20114 multiple times until the color histogram loss function converges, completing the training and constructing the color modulation module.

[0029] Optionally, step S2012 includes:

[0030] Step S20121: Obtain a glass image sample, input the glass image sample into the identification module, the identification module calculates the first true or false value of the glass image sample, and calculates the first loss function L D ;

[0031] Step S20122: receiving a target image sample, which is a glass image sample, performing color information modulation on the target image sample to obtain color control information to be injected; generating a glass sample color migration image at least based on the received glass image sample and the color control information to be injected;

[0032] Step S20123: Input the glass sample color migration image into the identification module, and the identification module calculates the second true and false value of the glass sample color migration image; at the same time, calculates the second loss function L G , calculate the color loss function L between the color histogram of the glass sample color migration image and the sample color histogram of the target image sample color , calculate the content similarity loss function L between the glass sample color migration image and the glass image sample content , the total loss function L of the glass image color migration system;

[0033] The first loss function L D , the second loss function L G , color loss function L color , content similarity loss function L content , the total loss function L of the glass image color migration system is:

[0034]

[0035] L G =-E z～p(z),noise～p(noise),style～p(style) [logD(G(z,noise,style,HisEncoder(hist(x)))]

[0036]

[0037] L content =||LDC(y -3 )-LDC(y -1 )||2

[0038] L=L D +L G +L color +L content

[0039] Where: x represents the glass image sample;

[0040] represents the expected function, which is related to x;

[0041] D(x) represents the first authenticity value calculated by the glass image sample through the identification module D;

[0042] E z～p(z),noise～p(noise),style～p(style) Represents the expected function, which is related to the tensor data z, noise information noise, and style information style;

[0043] z~p(z) represents the latent variable distribution of random tensor data;

[0044] noise~p(noise) represents the latent variable distribution of random noise information;

[0045] style~p(style) represents the latent variable distribution of random style information;

[0046] hist(x) represents the color histogram of the glass image sample;

[0047] HisEncoder(hist(x)) means converting the color histogram information of the glass image sample into color control information;

[0048] G(z,noise,style,HisEncoder(hist(x)) represents the glass sample color migration image generated by the generation module G by passing the random tensor data z, random style information style, random noise information noise and color control information;

[0049] D(G(z,noise,style,HisEncoder(hist(x))) represents the second true-false value calculated by the identification module D through the generated glass sample color migration image;

[0050] hist(G(z,noise,style,HisEncoder(hist(x))) represents the color histogram of the glass sample color migration image;

[0051] m is a positive integer, m=1, 2, 3, ..., n;

[0052] ||.||2 represents the two-norm;

[0053] H(.) represents the Hellinger distance function;

[0054] y -3 represents the glass color migration image generated by the Pth upsampling unit;

[0055] y -1 Represents the actual glass color migration image;

[0056] LDC represents a lightweight dense convolutional neural network for edge detection;

[0057] Step S20124: Use the optimizer to optimize the total loss function L, and repeat steps S20121, S20122, and S20123 multiple times until the total loss function L converges, completing the construction of the glass image color migration model.

[0058] Optionally, step S20122 includes:

[0059] Step S201221: taking the glass image sample as the target image sample, converting the target image sample into a sample color histogram according to the color modulation module, and modulating the sample color histogram to obtain the color control information required to be injected;

[0060] Step S201222: the first upsampling unit, the second upsampling unit, ..., the jth upsampling unit, ..., and the Pth upsampling unit are respectively used to obtain the first tensor data, the second tensor data, ..., the jth tensor data, ..., and the Pth tensor data, wherein the jth upsampling unit includes: a jth modulation subunit and a jth convolution activation unit, the jth modulation subunit is used to receive and modulate the image style information and the noise information, and the jth convolution activation unit is used to perform convolution fusion activation on the modulated image style information and the noise information to obtain the jth tensor data, j is a positive integer, j=1, 2, ..., P;

[0061] Step S201223: Generate the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image respectively based on at least the color control information to be injected and the first tensor data, at least the color control information to be injected and the second tensor data, ..., at least the color control information to be injected and the jth tensor data, ..., and at least the color control information to be injected and the Pth tensor data; fuse the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image to generate an actual glass color migration image.

[0062] In a third aspect, the present invention application provides a glass image color migration processing device, including a memory and a processor that are communicatively connected, wherein the memory is used to store a computer program, and the processor is used to read the computer program and execute the glass image color migration method as described in any one of the second aspects.

[0063] In a fourth aspect, the present invention application provides a computer-readable storage medium having instructions stored thereon, and when the instructions are run on a computer, the glass image color migration method as described in any one of the second aspects is executed.

[0064] (1) In the present invention, the color modulation module modulates the color information of the actual glass image, and the generation module can generate a large amount of image sample data with different color distributions based on the injected color control information. This can effectively expand the diversity of the image sample data set. When the existing glass defect image samples are insufficient, more variants can be generated through the glass image color migration technology to make up for the deficiency of insufficient sample data. By generating a rich image sample data set, the training data coverage of the subsequent glass quality monitoring model is optimized and improved, providing more sample data for the subsequent glass quality monitoring model.

[0065] (2) In the present invention application, by effectively generating different glass (defect) image color migration images, the diversity of the glass quality monitoring model training sample data is enhanced, thereby improving the generalization ability of the glass quality monitoring model. Even in different environments and conditions, the glass quality monitoring model can maintain good performance, thereby eliminating or reducing the impact of the surrounding color changes on image detection. Regardless of how the external ambient light or background color changes, the glass quality monitoring model can detect glass image defects based on the corresponding generated different glass image color migration images, thereby ensuring the stability and accuracy of the glass quality monitoring model under different environmental conditions, optimizing and improving the robustness and overall performance of the glass quality monitoring model, and being able to stably perform high-precision quality detection in a changing actual environment, showing significant technical advantages in actual glass image quality detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A schematic diagram of the composition of the glass image color migration system applied for by the present invention;

[0068] Figure 2 A schematic diagram of the composition of the color modulation unit of the present invention;

[0069] Figure 3 A schematic diagram of the composition of the color injection unit and the generation unit of the present invention;

[0070] Figure 4 A schematic diagram of the process of the glass image color migration method applied for by the present invention;

[0071] Figure 5 A schematic diagram of a sample of defect images of a black mobile phone screen glass applied for by the present invention;

[0072] Figure 6 Schematic diagram of the comparison of the target image sample and the color migration effect of different glass color migration models applied in the present invention ( Figure 6 (a) is a schematic diagram of a target image sample. Figure 6 (b) is a schematic diagram of the color migration effect of the HistoGAN glass color migration model. Figure 6 (c) is a schematic diagram of the color migration effect of the DWGAN glass color migration model and Figure 6 (d) Schematic diagram of the color migration effect of the glass color migration model applied in the present invention).

[0073] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0075] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside" and the like indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention. In order to make the purpose, technical solution and advantages of the present invention clearer, the implementation methods of the present invention will be further described in detail below in conjunction with the drawings.

[0076] Glass Image Color Migration System

[0077] like Figures 1 to 3 As shown, in the first aspect, a glass image color migration system comprises:

[0078] Image input module 100, glass image color migration model 200, image output module 300;

[0079] The image input module 100 acquires the actual glass image; the glass image color migration model receives the actual glass image and performs color migration on the actual glass image to generate an actual glass color migration image; the image output module outputs the actual glass color migration image;

[0080] The glass image color migration model 200 includes: a color modulation module 201 and a generation module; the color modulation module receives a target image, modulates the color information of the target image, and obtains the required injected color control information; the generation module generates an actual glass color migration image based on at least the actual glass image and the required injected color control information.

[0081] In the glass image color migration system of the present invention, first, the color modulation module modulates the color information of the target object to obtain the required injected color control information, and the generation module can generate a large number of image sample data with different color distributions based on the injected color control information, which can effectively expand the diversity of the image sample data set. In the case of insufficient existing glass defect image samples, more sample variants can be generated through the glass image color migration technology to make up for the defect of insufficient sample data. By generating a rich image sample data set, the training data coverage of the subsequent glass quality monitoring model is optimized and improved, and more sample data is provided for the subsequent glass quality monitoring model. In addition, the actual glass image and the glass image color migration model correspond to each other and adapt to each other. By effectively generating different glass (defect) images Color migration images enhance the diversity of sample data for subsequent glass quality monitoring model training, thereby improving the generalization ability of subsequent glass quality monitoring models. Even in different environments and conditions, the glass quality monitoring model can maintain good performance, thereby eliminating or reducing the impact of surrounding color changes on image detection. Regardless of how the external ambient light or background color changes, the glass quality monitoring model can detect glass image defects based on the corresponding generated different glass image color migration images, ensuring the stability and accuracy of the glass quality monitoring model under different environmental conditions, optimizing and improving the robustness and overall performance of the glass quality monitoring model, and being able to stably perform high-precision quality inspections in a changing actual environment, showing significant technical advantages in actual glass image quality inspections.

[0082] Optionally, the color modulation module 201 includes a color histogram unit 2011, a modulation unit 2012 and a demodulation unit 2013, the modulation unit 2012 includes multiple downsampling subunits, a first convolution subunit one, and a first convolution subunit two, each of which includes a convolution subunit one and a batch normalization subunit one; the demodulation unit 2013 includes multiple upsampling subunits, a second convolution subunit and an adaptive pooling subunit, each of which includes a convolution subunit two and a batch normalization subunit two; the color histogram unit converts the target image into a target color histogram, the modulation unit modulates the target color histogram to obtain the color control information to be injected; the demodulation unit demodulates the color control information to be injected to obtain a demodulated color intuitive map.

[0083] In some embodiments, the generation module can be a generator in the existing adversarial generative network technology field, and the target image can be an actual glass image, or any color image or style image, etc. Those skilled in the art can choose according to actual conditions, and the present invention application does not impose any special restrictions on this.

[0084] In some embodiments, the color histogram is a two-dimensional histogram that projects the color of the target image into a logarithmic chromaticity space. This 2D histogram is parameterized by uv to convey the color information of the target image, and is more compact than a typical 3D histogram defined in RGB space. The logarithmic chromaticity space is defined by the intensity of one channel and normalized by the other two channels. It selects one channel at a time to construct a histogram. RGB can construct three different histograms, which are combined into a histogram feature H. The color intuitive map is a tensor data of h*h*3.

[0085] In some embodiments, the number of downsampling subunits is 4 or 5 or more, and the number of upsampling subunits is 4 or 5 or more. Those skilled in the art can set the number of downsampling subunits and upsampling subunits according to actual needs, and the present invention application does not impose any limitation on this.

[0086] In some embodiments, the color information modulation module includes a modulation unit and a demodulation unit, and the modulation unit may include 4 downsampling subunits, a first convolution subunit 1 and a first convolution subunit 2, and each downsampling subunit includes a convolution and batch normalization module. Specifically, the 4 downsampling subunits, the first convolution subunit 1 and the first convolution subunit 2 are correspondingly connected, and the target color histogram is modulated by the first convolution subunit 1 of the modulation unit, the 4 downsampling subunits and the first convolution subunit 2 to obtain the color control information required for injection.

[0087] In some embodiments, the demodulation unit includes 4 upsampling subunits, a second convolution subunit, and an adaptive pooling subunit. Specifically, the 4 upsampling subunits, the second convolution subunit, and the adaptive pooling subunit are correspondingly connected, and the color control information to be injected is demodulated by the adaptive pooling subunit, the 4 upsampling subunits, and the second convolution subunit of the demodulation unit, and then demodulated to obtain a demodulated color intuitive map.

[0088] Optionally, the generation module includes: a 1st upsampling unit, a 2nd upsampling unit, ..., a jth upsampling unit, ..., a Pth upsampling unit, the 1st upsampling unit, the 2nd upsampling unit, ..., the jth upsampling unit, ..., the Pth upsampling unit are respectively used to obtain the 1st tensor data, the 2nd tensor data, ..., the jth tensor data, ..., the Pth tensor data, wherein the jth upsampling unit includes: a jth modulation subunit and a jth convolution activation unit, the jth modulation subunit is used to receive and modulate the image style information and the noise information, the jth convolution activation unit is used to perform convolution fusion activation on the modulated image style information and the noise information to obtain the jth tensor data, j is a positive integer, j=1, 2, ..., P;

[0089] In some embodiments, the number P of upsampling units in the generation module can be determined by the size of the actual glass color migration image, and the specific calculation formula is as follows:

[0090] P = log2(image_size)-1

[0091] Among them, image_size is the actual glass color migration image size, and P is the number of upsampling units.

[0092] Therefore, the generation module can adaptively select the appropriate number of sampling units according to the sizes of different actual glass images, and then the generation module can more accurately process actual glass images of different sizes, ensuring that the quality of the generated actual glass image sampling images matches the actual needs.

[0093] In some embodiments, by modulating the image style information and noise information through the modulation subunit, for example, the image style information includes information such as the texture and shape of the image, and a variety of image variants containing image styles and noise can be generated, thereby enriching the training data sample set and increasing the actual image sample nature of the glass.

[0094] In some embodiments, the convolution activation unit is used to perform 3×3 convolution fusion and LReLU function activation on the actual glass image and the modulated image style and noise information, which can better capture the details and features of the actual glass image and enhance the quality of the actual glass image. It can be used to subsequently improve the diversity of glass image samples to meet the sample data requirements of the glass quality monitoring model.

[0095] Optionally, the generation module also includes: a color injection unit 202 and a generation unit 203, wherein the color injection unit 202 generates a first glass color migration image, a second glass color migration image, ..., a jth glass color migration image, ..., a Pth glass color migration image, respectively, according to at least the color control information required to be injected and the first tensor data, at least the color control information required to be injected and the second tensor data, ..., at least the color control information required to be injected and the jth tensor data, ..., at least the color control information required to be injected and the Pth tensor data, wherein the color injection unit includes a first convolution activation unit 2021, a second convolution activation unit 2022, and a fusion convolution unit 2023, wherein the first convolution activation unit and the second convolution activation unit are arranged in parallel, and the first convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the Pth convolution activation unit are arranged in parallel, and the first convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the Pth convolution activation unit are arranged in parallel, and the first convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the Pth convolution activation unit are arranged in parallel, and the first convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the second convolution activation unit are arranged in parallel, and the second convolution activation unit and the second convolution activation unit are arranged in parallel, and the first ... A convolution activation unit includes multiple convolution units 1 and multiple activation units 1, and a second convolution activation unit includes multiple convolution units 2 and multiple activation units 2; the first convolution activation unit performs convolution fusion activation at least according to the received color control information required for injection and the jth tensor data to obtain the jth color injection data 1; the second convolution activation unit performs convolution fusion activation at least according to the received color control information required for injection and the jth tensor data to obtain the jth color injection data 2; the fusion convolution unit performs fusion convolution on the jth color injection data 1 and the jth color injection image data 2 to obtain the jth glass color migration image; the generation unit fuses the 1st glass color migration image, the 2nd glass color migration image, ..., the jth glass color migration image, ..., the Pth glass color migration image to generate an actual glass color migration image.

[0096] In some embodiments, the number of convolution units 1 and 2 can be 2 or 3 or more, and correspondingly, the number of activation units 1 and 2 can be 2 or 3 or more. The number of convolution units 1 and 2 can be connected correspondingly, and the number of convolution units 2 and 2 can be connected correspondingly. Those skilled in the art can set the number of convolution units 1 and 2, activation units 1 and 2 according to actual needs, and the present invention application does not impose any limitation on this.

[0097] In some embodiments, the first convolution activation unit performs the first convolution fusion activation and the second convolution fusion activation at least based on the received color control information required for injection and the j-th tensor data, to obtain the first color injection data, wherein the first convolution fusion and the second convolution fusion can both be 3×3 convolution fusion; at the same time, the second convolution activation unit also performs the first convolution fusion activation and the second convolution fusion activation at least based on the received color control information required for injection and the actual glass image sampling image to obtain the second color injection image data, wherein the first convolution fusion and the second convolution fusion can both be 3×3 convolution fusion.

[0098] In some embodiments, the first convolution activation unit and the second convolution activation unit work in parallel, and the jth color injection data 1 and the jth color injection data 2 are generated respectively by performing multiple convolution fusion activations on the jth tensor data and the color control information, and the fusion convolution unit performs fusion convolution on the jth color injection data 1 and the jth color injection image data 2 to obtain the jth glass color migration image, and the fusion convolution unit includes 1×1 convolution. Thus, through the above parallel processing and fusion method, the color control information and the tensor data can be fused multiple times to ensure that the color control information is fully injected.

[0099] In some embodiments, the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., the Pth glass color migration image are fused to generate an actual glass color migration image, which can reduce detail loss or unnatural color transition that may occur during the color migration of the actual glass image, effectively preserve the original features of the actual glass image, and at the same time achieve accurate migration of the target color of the actual glass image, thereby ensuring the quality of the generated actual glass color migration image.

[0100] Glass image color transfer method

[0101] like Figure 4 As shown, in a second aspect, a glass image color migration method comprises:

[0102] Step S100: acquiring an actual glass image;

[0103] Step S200: receiving an actual glass image, and performing color migration on the actual glass image using a glass image color migration model to generate an actual glass color migration image;

[0104] It should be noted that a glass image color migration method applied for in the present invention may adopt or not adopt any of the glass image color migration systems in the first aspect mentioned above. When adopting a glass image color migration system including any of the first aspect, it correspondingly also includes: all the technical problems, technical solutions and technical effects recorded in any of the glass image color migration systems in the first aspect, and the present invention application will not be repeated here.

[0105] Specifically, step S200 includes:

[0106] Step S201: constructing a glass image color migration model;

[0107] Step S202: receiving a target image, modulating the color information of the target image using a glass image color transfer model, obtaining the required injected color control information, and generating an actual glass color transfer image at least based on the actual glass image and the required injected color control information.

[0108] In the glass image color migration method of the present invention, a glass image color migration model is constructed in step S201, and the color migration of the actual glass image in step S100 is performed using the glass image color migration model to generate an actual glass image color image. Specifically, the method includes: receiving an actual glass image and a target image, and modulating the color information of the target image using the glass image color migration model to obtain the color control information required to be injected; and generating an actual glass color migration image based on at least the actual glass image and the color control information required to be injected.

[0109] Therefore, firstly, the color information of the target image can be modulated accordingly according to the target image to obtain the required injected color control information, and a large number of image sample data with different color distributions can be generated according to the injected color control information, which can effectively expand the diversity of the image sample data set. When the existing glass defect image samples are insufficient, more sample data can be generated through the glass image color migration technology to make up for the defect of insufficient sample data. By generating a rich image sample data set, the training data coverage of the glass quality monitoring model is optimized and improved, and more sample data is provided for the glass quality monitoring model. In addition, the actual glass image and the glass image color migration model are adapted to each other, and different glass (defective) images can be effectively generated. The color migration image of the image enhances the diversity of the sample data for training the glass quality monitoring model, thereby improving the generalization ability of the glass quality monitoring model. Even in different environments and conditions, the glass quality monitoring model can maintain good performance, thereby eliminating or reducing the impact of the color changes of the surrounding environment on image detection. No matter how the external ambient light or background color changes, the glass quality monitoring model can generate different glass image color migration images to detect glass image defects, ensuring the stability and accuracy of the glass quality monitoring model under different environmental conditions, optimizing and improving the robustness and overall performance of the glass quality monitoring model, and being able to stably perform high-precision quality inspections in a changing actual environment.

[0110] Optionally, the glass image color migration model includes a color modulation module, a generation module and an identification module; step S201 includes:

[0111] Step S2011: training and constructing a color modulation module;

[0112] Specifically, step S2011 includes:

[0113] Step S20111: obtaining an ImageNet image dataset, where the ImageNet image dataset includes a 1st ImageNet image, a 2nd ImageNet image, ..., an ith ImageNet image, ..., an Nth ImageNet image;

[0114] Step S20112: converting the 1st ImageNet image, the 2nd ImageNet image, ..., the i-th ImageNet image, ..., the N-th ImageNet image into the 1st color histogram, the 2nd color histogram, ..., the i-th color histogram, ..., the N-th color histogram;

[0115] Step S20113: modulating the first color histogram, the second color histogram, ..., the i-th color histogram, ..., and the N-th color histogram respectively to obtain the first color control information, the second color control information, ..., the i-th color control information, ..., and the N-th color control information; demodulating the first color control information, the second color control information, ..., the i-th color control information, ..., and the N-th color control information to obtain the first demodulated color histogram, the second demodulated color histogram, ..., the i-th demodulated color histogram, ..., and the N-th demodulated color histogram;

[0116] Step S20114: calculating the sum of the mean square errors of the first demodulated color histogram, the second demodulated color histogram, ..., the i-th demodulated color histogram, ..., the N-th demodulated color histogram and the first color histogram, the second color histogram, ..., the i-th color histogram, ..., the N-th color histogram respectively as a color histogram loss function;

[0117] Step S20115: Use the optimizer to optimize the color histogram loss function, and repeat steps S20112, S20113, and S20114 multiple times until the color histogram loss function converges, completing the training and constructing the color modulation module.

[0118] In step S2011, the ImageNet image dataset can be used as the target image dataset. By acquiring and processing the ImageNet image dataset, different ImageNet images are converted into color histograms and then modulated and demodulated, multiple demodulated color histograms can be obtained. The modulation and demodulation process and the application of the loss function are iteratively optimized multiple times, and the difference between the demodulated color histogram and the original color histogram is gradually reduced, thereby achieving an ideal image color change effect. The color modulation module training and construction can effectively absorb rich color information, which lays a good foundation for subsequently increasing the diversity of glass image samples and injecting color control information.

[0119] Step S2012: training and constructing a glass image color migration model according to the color modulation module;

[0120] Specifically, step S2012 includes:

[0121] Step S2012 includes:

[0122] Step S20121: Obtain a glass image sample, input the glass image sample into the identification module, the identification module calculates the first true or false value of the glass image sample, and calculates the first loss function L D , the first loss function L D for:

[0123]

[0124] represents the expected function, which is related to x;

[0125] D(x) represents the first true-false value calculated by the glass image sample through the identification module D;

[0126] E z～p(z),noise～p(noise),style～p(style) Represents the expected function, which is related to z, noise, and style.

[0127] z~p(z) represents the latent variable distribution of random tensor data;

[0128] noise~p(noise) represents the latent variable distribution of random noise information;

[0129] style~p(style) represents the latent variable distribution of random style information;

[0130] hist(x) represents the color histogram of the glass image sample;

[0131] HisEncoder(hist(x)) means modulating the color histogram information of the glass image sample into color control information;

[0132] G(z,noise,style,HisEncoder(hist(x)) represents the color migration image generated by the generation module G by adding random tensor data z, random style information style, random noise information noise, and color control information;

[0133] D(G(z,noise,style,HisEncoder(hist(x))) represents the second truth value calculated by the identification module D for the generated color migration image.

[0134] Step S20122: receiving a target image sample, which is a glass image sample, performing color information modulation on the target image sample to obtain color control information to be injected; generating a glass sample color migration image at least based on the received glass image sample and the color control information to be injected;

[0135] Specifically, step S20122 includes:

[0136] Step S201221: taking the glass image sample as the target image sample, converting the target image sample into a sample color histogram according to the color modulation module, and modulating the sample color histogram to obtain the color control information required to be injected;

[0137] Step S201222: the first upsampling unit, the second upsampling unit, ..., the jth upsampling unit, ..., and the Pth upsampling unit are respectively used to obtain the first tensor data, the second tensor data, ..., the jth tensor data, ..., and the Pth tensor data, wherein the jth upsampling unit includes: a jth modulation subunit and a jth convolution activation unit, the jth modulation subunit is used to receive and modulate the image style information and the noise information, and the jth convolution activation unit is used to perform convolution fusion activation on the modulated image style information and the noise information to obtain the jth tensor data, j is a positive integer, j=1, 2, ..., P;

[0138] Step S201223: Generate the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image respectively based on at least the color control information to be injected and the first tensor data, at least the color control information to be injected and the second tensor data, ..., at least the color control information to be injected and the jth tensor data, ..., and at least the color control information to be injected and the Pth tensor data; fuse the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image to generate an actual glass color migration image.

[0139] In the present application, the color injection unit may include a first convolution activation unit, a second convolution activation unit, and a fused convolution unit. The first convolution activation unit and the second convolution activation unit are arranged in parallel. The first convolution activation unit includes multiple convolution units one and multiple activation units one, and the second convolution activation unit includes multiple convolution units two and multiple activation units two. The first convolution activation unit performs convolution fusion activation at least based on the color control information required to be injected and the jth tensor data to obtain the jth color injection data one. The second convolution activation unit performs convolution fusion activation at least based on the color control information required to be injected and the jth tensor data to obtain the jth color injection data two. The fused convolution unit performs fusion convolution on the jth color injection data one and the jth color injection image data two to obtain the jth glass color migration image. By analogy, the color injection unit can generate the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image respectively according to at least the color control information to be injected and the first tensor data, at least the color control information to be injected and the second tensor data, ..., at least the color control information to be injected and the jth tensor data, ..., and at least the color control information to be injected and the Pth tensor data; then, the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image are fused to generate the actual glass color migration image.

[0140] Step S20123: Input the glass sample color migration image into the identification module, and the identification module calculates the second true and false value of the glass sample color migration image; at the same time, calculates the second loss function L G , calculate the color loss function L between the color histogram of the glass sample color migration image and the sample color histogram of the target image sample color , calculate the content similarity loss function L between the glass sample color migration image and the glass image sample content , the total loss function L of the glass image color migration system;

[0141] In some embodiments, the second loss function L G , color loss function L color , content similarity loss function L content , the total loss function L of the glass image color migration system is:

[0142] L G =-

[0143] E z～p(z),noise～p(noise),style～p(style) [logD(G(z,noise,style,HisEncoder(hist(x)))]

[0144]

[0145] L content =||LDC(y -3 )-LDC(y -1 )||2

[0146] L=L D +L G +L color +L content

[0147] Where: x represents the glass image sample;

[0148] represents the expected function, which is related to x;

[0149] D(x) represents the first authenticity value calculated by the glass image sample through the identification module D;

[0150] E z～p(z),noise～p(noise),style～p(style) Represents the expected function, which is related to the tensor data z, noise information noise, and style information style.

[0151] z~p(z) represents the latent variable distribution of random tensor data;

[0152] noise~p(noise) represents the latent variable distribution of random noise information;

[0153] style~p(style) represents the latent variable distribution of random style information;

[0154] hist(x) represents the color histogram of the glass image sample;

[0155] HisEncoder(hist(x)) means converting the color histogram information of the glass image sample into color control information;

[0156] G(z,noise,style,HisEncoder(hist(x)) represents the glass sample color migration image generated by the generation module G by passing the random tensor data z, random style information style, random noise information noise and color control information;

[0157] D(G(z,noise,style,HisEncoder(hist(x))) represents the second true-false value calculated by the identification module D through the generated glass sample color migration image;

[0158] hist(G(z,noise,style,HisEncoder(hist(x))) represents the color histogram of the glass sample color migration image;

[0159] m is a positive integer, m=1, 2, 3, ..., n;

[0160] y -3 represents the P-th glass color migration image generated by the P-th upsampling unit;

[0161] y -1 Represents the actual glass color migration image;

[0162] ||.||2 represents the two-norm;

[0163] H(.) represents the Hellinger distance function;

[0164] LDC (Lightweight Dense CNN for Edge Detection, referred to as LDC) represents a lightweight dense convolutional neural network for edge detection.

[0165] In some embodiments, the identification module can be a discriminator in the existing adversarial generative network technology field. The identification module includes multiple downsampling modules and a fully connected module, the downsampling module includes a convolution module and an activation module, the convolution module is connected to the activation module, the downsampling is connected to the fully connected module, and the identification module can output a true or false value between 0 and 1 to indicate whether the image is true or false.

[0166] In some embodiments, the color loss L color Also known as Hellinger distance loss, Hellinger distance is an indicator used to measure the difference between two probability distributions. It is based on the square root of Hellinger distance and is mainly used to compare the similarity of two distributions. It is widely used in statistics and machine learning.

[0167] In some embodiments, LDC is a lightweight dense convolutional neural network for edge detection that can achieve the highest score compared to other lightweight models, and only requires 4% of the parameters to maintain the same level as heavyweight models. Using LDC can better display information about the image content.

[0168] Step S20125: Use the optimizer to optimize the total loss function L, and repeat steps S20121, S20122, and S20123 multiple times until the total loss function L converges, completing the construction of the glass image color migration model.

[0169] Therefore, by training a color modulation module based on multiple glass image samples and training and constructing a glass image color migration model based on the color modulation module, the actual glass image and the glass image color migration model can be adapted to each other.

[0170] Step S200 may further include step S300: displaying and outputting an actual glass color migration image.

[0171] In some embodiments, the actual glass color migration image can be displayed and output. For example, a display screen can be used to display and output the actual glass color migration image. The user can intuitively perceive the impact of color migration on the glass image to ensure that the actual color migration effect meets expectations. The parameters of the glass image color migration system can also be further adjusted when necessary, which also lays a good image data foundation for the subsequent glass image quality monitoring model, is conducive to the subsequent monitoring of potential defects in the actual glass image, and improves the working efficiency of the glass image quality monitoring model.

[0172] Building a Glass Image Color Migration System Example

[0173] like Figure 5 , Figure 6 As shown in the figure, the public glass image sample dataset: widescreen mobile phone screen glass defect image sample dataset, the widescreen mobile phone screen glass defect image sample dataset contains 186 black mobile phone screen glass defect image samples, Figure 6 (a) is a target image sample, and there may be 6 target image samples.

[0174] The HistoGAN glass color migration model, the DWGAN glass color migration model, and the glass color migration model of the present invention were compared and tested. The models were trained using a GeForce RTX 3080Ti graphics card with a training iteration of 10k. Figure 6 (b) shows the glass color migration image effect of the HistoGAN glass color migration model. Figure 6(c) is the glass color migration image effect of the DWGAN glass color migration model. There is basically no obvious color migration. The glass color migration model of the present invention has a color migration effect. At the same time, the Hellinger distance result can be used as a parameter to evaluate the color migration effect of the glass image. The smaller the Hellinger distance, the better the glass color migration effect. The Hellinger distance of the HistoGAN glass color migration model is 0.6801, the Hellinger distance of the DWGAN glass color migration model is 0.6303, and the Hellinger distance of the glass color migration model of the present invention is 0.4777. Therefore, by comparing the Hellinger distance results of the HistoGAN glass color migration model, the DWGAN glass color migration model and the glass color migration system constructed by the present invention, it can be seen that the glass color migration system constructed by the present invention has a better glass color migration effect.

[0175] Glass image color migration processing device

[0176] In a third aspect, the present invention application provides a glass image color migration processing device, comprising a memory and a processor that are communicatively connected, wherein the memory is used to store a computer program, and the processor is used to read the computer program and execute the steps in the glass image color migration method as described in any one of the second aspects.

[0177] In some embodiments, a glass image color migration processing device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the glass image color migration methods described in the second aspect are implemented.

[0178] Computer readable storage medium

[0179] In a fourth aspect, the present invention application provides a computer-readable storage medium having instructions stored thereon, and when the instructions are run on a computer, the steps in the glass image color migration method as described in any one of the second aspects are executed.

[0180] Those skilled in the art will understand that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0181] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0182] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0183] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0184] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0185] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A glass image color migration system, characterized in that: include: Image input module, glass image color migration model, image output module; Image input module, to obtain the actual glass image; A glass image color transfer model receives an actual glass image and performs color transfer on the actual glass image to generate an actual glass color transfer image; Image output module, outputting actual glass color migration image; The glass image color migration model includes: a color modulation module and a generation module; the color modulation module receives a target image, modulates the color information of the target image, and obtains the required injected color control information; the generation module generates an actual glass color migration image based on at least the actual glass image and the required injected color control information.

2. A glass image color migration system according to claim 1, characterized in that: The color modulation module includes a color histogram unit, a modulation unit and a demodulation unit. The modulation unit includes multiple downsampling subunits, a first convolution subunit one, and a first convolution subunit two. Each downsampling subunit includes a convolution subunit one and a batch normalization subunit one. The demodulation unit includes multiple upsampling subunits, a second convolution subunit and an adaptive pooling subunit. Each upsampling subunit includes a convolution subunit two and a batch normalization subunit two. The color histogram unit converts the target image into a target color histogram. The modulation unit modulates the target color histogram to obtain the color control information to be injected. The demodulation unit demodulates the color control information to be injected to obtain a demodulated color intuitive map.

3. A glass image color migration system according to claim 2, characterized in that: The generation module includes: a 1st upsampling unit, a 2nd upsampling unit, ..., a jth upsampling unit, ..., a Pth upsampling unit, the 1st upsampling unit, the 2nd upsampling unit, ..., the jth upsampling unit, ..., the Pth upsampling unit are respectively used to obtain a 1st tensor data, a 2nd tensor data, ..., a jth tensor data, ..., a Pth tensor data; wherein the jth upsampling unit includes: a jth modulation subunit and a jth convolution activation unit, the jth modulation subunit is used to receive and modulate image style information and noise information, the jth convolution activation unit is used to convolutionally fuse and activate the modulated image style information and noise information to obtain the jth tensor data, j is a positive integer, j=1, 2, ..., P; The generation module also includes: a color injection unit and a generation unit, wherein the color injection unit generates the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., the Pth glass color migration image, respectively, according to at least the color control information to be injected and the first tensor data, at least the color control information to be injected and the second tensor data, ..., at least the color control information to be injected and the jth tensor data, ..., at least the color control information to be injected and the Pth tensor data, wherein the color injection unit includes a first convolution activation unit, a second convolution activation unit, and a fusion convolution. The first convolution activation unit and the second convolution activation unit are arranged in parallel, the first convolution activation unit includes a plurality of convolution units 1 and a plurality of activation units 1, and the second convolution activation unit includes a plurality of convolution units 2 and a plurality of activation units 2; the first convolution activation unit performs convolution fusion activation at least according to the received color control information required for injection and the jth tensor data to obtain the jth color injection data 1; the second convolution activation unit performs convolution fusion activation at least according to the received color control information required for injection and the jth tensor data to obtain the jth color injection data 2; the fusion convolution unit performs fusion convolution on the jth color injection data 1 and the jth color injection image data 2 to obtain the jth glass color migration image; The generating unit fuses the first glass color transition image, the second glass color transition image, ..., the jth glass color transition image, ..., the Pth glass color transition image to generate an actual glass color transition image.

4. A glass image color migration method, using a glass image color migration system according to any one of claims 1 to 3, characterized in that: include: Step S100: acquiring an actual glass image; Step S200: receiving an actual glass image, and performing color migration on the actual glass image using a glass image color migration model to generate an actual glass color migration image; Step S200 includes: Step S201: constructing a glass image color migration model; Step S202: receiving a target image, modulating the color information of the target image using a glass image color transfer model, obtaining the required injected color control information, and generating an actual glass color transfer image at least based on the actual glass image and the required injected color control information.

5. The glass image color migration method according to claim 4, characterized in that: The glass image color migration model includes a color modulation module, a generation module and an identification module; step S201 includes: Step S2011: training and constructing a color modulation module; Step S2012: Based on the color modulation module, a glass image color migration model is trained and constructed.

6. The glass image color migration method according to claim 4, characterized in that: Step S2011 includes: Step S20111: Obtain an ImageNet image dataset, where the ImageNet image dataset includes a 1st ImageNet image, a 2nd ImageNet image, ..., an ith ImageNet image, ..., an Nth ImageNet image; Step S20112: converting the 1st ImageNet image, the 2nd ImageNet image, ..., the i-th ImageNet image, ..., the N-th ImageNet image into the 1st color histogram, the 2nd color histogram, ..., the i-th color histogram, ..., the N-th color histogram; Step S20113: modulating the first color histogram, the second color histogram, ..., the i-th color histogram, ..., and the N-th color histogram respectively to obtain the first color control information, the second color control information, ..., the i-th color control information, ..., and the N-th color control information; demodulating the first color control information, the second color control information, ..., the i-th color control information, ..., and the N-th color control information to obtain the first demodulated color histogram, the second demodulated color histogram, ..., the i-th demodulated color histogram, ..., and the N-th demodulated color histogram; Step S20114: calculating the sum of the mean square errors of the first demodulated color histogram, the second demodulated color histogram, ..., the i-th demodulated color histogram, ..., the N-th demodulated color histogram and the first color histogram, the second color histogram, ..., the i-th color histogram, ..., the N-th color histogram respectively as a color histogram loss function; Step S20115: Use the optimizer to optimize the color histogram loss function, and repeat steps S20112, S20113, and S20114 multiple times until the color histogram loss function converges, completing the training and constructing the color modulation module.

7. The glass image color migration method according to claim 6, characterized in that: Step S2012 includes: Step S20121: Obtain a glass image sample, input the glass image sample into the identification module, the identification module calculates the first true or false value of the glass image sample, and calculates the first loss function L D ; Step S20122: receiving a target image sample, which is a glass image sample, performing color information modulation on the target image sample to obtain color control information to be injected; generating a glass sample color migration image at least according to the received glass image sample and the color control information to be injected; Step S20123: Input the glass sample color migration image into the identification module, and the identification module calculates the second true and false value of the glass sample color migration image; at the same time, calculates the second loss function L G , calculate the color loss function L between the color histogram of the glass sample color migration image and the sample color histogram of the target image sample color , calculate the content similarity loss function L between the glass sample color migration image and the glass image sample content , the total loss function L of the glass image color migration system; The first loss function L D , the second loss function L G , color loss function L color , content similarity loss function L content , the total loss function L of the glass image color migration system is: L G ＝- E z～p(z),noise～p(noise),style～p(style) [logD(G(z,noise,style,HisEncoder(hist(x)))] <h2 style=";text-align:left;direction:ltr">L<h2 style=";text-align:left;direction:ltr"> content <h2 style=";text-align:left;direction:ltr"> =||LDC(y<h2 style=";text-align:left;direction:ltr"> -3 <h2 style=";text-align:left;direction:ltr"> )-LDC(y<h2 style=";text-align:left;direction:ltr"> -1 <h2 style=";text-align:left;direction:ltr"> )||2 L=L D +L G +L color +L content Where: x represents the glass image sample; represents the expected function, which is related to x; D(x) represents the first authenticity value calculated by the glass image sample through the identification module D; E z～p(z),noise～p(noise),style～p(style) Represents the expected function, which is related to the tensor data z, noise information noise, and style information style; z~p(z) represents the latent variable distribution of random tensor data; noise~p(noise) represents the latent variable distribution of random noise information; style~p(style) represents the latent variable distribution of random style information; hist(x) represents the color histogram of the glass image sample; HisEncoder(hist(x)) means modulating the color histogram information of the glass image sample into color control information; G(z,noise,style,HisEncoder(hist(x)) represents the glass sample color migration image generated by the generation module G by passing the random tensor data z, random style information style, random noise information noise and color control information; D(G(z,noise,style,HisEncoder(hist(x))) represents the second true-false value calculated by the identification module D through the generated glass sample color migration image; hist(G(z,noise,style,HisEncoder(hist(x))) represents the color histogram of the glass sample color migration image; m is a positive integer, m=1, 2, 3, ..., n; y -3 represents the glass color migration image generated by the Pth upsampling unit; y -1 Represents the actual glass color migration image; ||.||2 represents the two-norm; H(.) represents the Hellinger distance function; LDC represents a lightweight dense convolutional neural network for edge detection; Step S20124: Use the optimizer to optimize the total loss function L, and repeat steps S20121, S20122, and S20123 multiple times until the total loss function L converges, completing the construction of the glass image color migration model.

8. The glass image color migration method according to claim 4, characterized in that: Step S20122 includes: Step S201221: taking the glass image sample as the target image sample, converting the target image sample into a sample color histogram according to the color modulation module, and modulating the sample color histogram to obtain the color control information required to be injected; Step S201222: the first upsampling unit, the second upsampling unit, ..., the jth upsampling unit, ..., and the Pth upsampling unit are respectively used to obtain the first tensor data, the second tensor data, ..., the jth tensor data, ..., and the Pth tensor data, wherein the jth upsampling unit includes: a jth modulation subunit and a jth convolution activation unit, the jth modulation subunit is used to receive and modulate the image style information and the noise information, and the jth convolution activation unit is used to perform convolution fusion activation on the modulated image style information and the noise information to obtain the jth tensor data, j is a positive integer, j=1, 2, ..., P; Step S201223: Generate the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image respectively based on at least the color control information to be injected and the first tensor data, at least the color control information to be injected and the second tensor data, ..., at least the color control information to be injected and the jth tensor data, ..., and at least the color control information to be injected and the Pth tensor data; fuse the first glass color migration image, the second glass color migration image, ..., the jth glass color migration image, ..., and the Pth glass color migration image to generate an actual glass color migration image.

9. A glass image color migration processing device, characterized in that: It comprises a memory and a processor which are communicatively connected, wherein the memory is used to store a computer program, and the processor is used to read the computer program and execute the glass image color migration method as described in any one of claims 4 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the glass image color migration method according to any one of claims 4 to 8 is executed.

Images