Quality enhancement method and device including clothing image data

Abstract

The invention discloses a quality enhancement method and device containing clothing image data. The method comprises the following steps: performing multi-modal content analysis on a to-be-processed image by using a preset artificial intelligence model to analyze figure features and clothing information in the image, and embedding the figure features and the clothing information into a submerged space; calling an image restoration model, performing diffusion processing on the image in the submerged space based on priori knowledge of image statistical characteristics, extracting structure information and texture details in the image, and performing block coding on the image; based on the image restoration model, taking image representation in a submerged space of the image restoration model as a starting point, and generating a first image through diffusion sampling; and detecting portrait features in the first image through multi-scale feature fusion and anchor point detection, and performing detail restoration of the portrait features based on adversarial training of a generator and a discriminator of the generative adversarial network to obtain an output image. According to the invention, deep optimization of the image data is realized.

Description

Technical Field

[0001] The present application relates to a key image data extraction technology including clothing image data, and more particularly to a quality enhancement method and device including clothing image data. Background Art

[0002] When generating clothing images, traditional methods often fail to achieve ideal model generation results due to low data quality, which is manifested in poor realism, texture, and clarity of the generated clothing images. Existing public clothing datasets usually have low resolution and fuzzy images. When high-resolution data within an enterprise is compressed to the resolution required for model training, the image quality further deteriorates and is not suitable as high-quality training image data. Therefore, the details of such datasets are unclear at low resolution, and the material is difficult to distinguish, resulting in unsatisfactory model training results and low quality of the output clothing images.

[0003] The more popular methods for improving the image quality of datasets mainly rely on super-resolution algorithms based on generative adversarial networks (GANs), such as Enhanced Super-Resolution Generative Adversarial Network (ESRGAN), Blind Super-Resolution Generative Adversarial Network (BSRGAN) and UltraSharp. Although these methods can improve image resolution to a certain extent, they have some significant defects, including the further loss of material and texture information. This type of algorithm has the problem of secondary image compression during the resolution enhancement process, resulting in the quality of the enhanced image being unclear and unable to meet the training requirements of the model. Summary of the invention

[0004] The present application provides a method and device for enhancing the quality of clothing image data to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present application, a quality enhancement method for clothing image data is provided, comprising:

[0006] Use the preset artificial intelligence model to perform multimodal content analysis on the processed image to analyze the information of the person features and clothing in the image and embed them into the latent space;

[0007] Call the image restoration model, based on the prior knowledge of the statistical characteristics of the image, perform diffusion processing on the image in the latent space, extract the structural information and texture details in the image, and encode the image in blocks;

[0008] Based on the image restoration model, taking the image representation in the latent space of the image restoration model as a starting point, and generating a first image through diffusion sampling;

[0009] The portrait features in the first image are detected through multi-scale feature fusion and anchor point detection, and the details of the portrait features are restored based on the adversarial training of the generator and the discriminator of the generative adversarial network to obtain an output image.

[0010] In some executable embodiments, before performing multimodal content analysis on the image to be processed, the method further includes:

[0011] The input image to be processed is subjected to bilinear interpolation scaling to improve the resolution of the image to be processed; wherein the formula of bilinear interpolation is:

[0012]

[0013] Among them, I(x1, y1), I(x2, y2), I(x2, y1), I(x1, y2) are the pixel values ​​of the adjacent pixels of the pixel to be bilinearly interpolated, and I(x, y) is the pixel value of the pixel to be bilinearly interpolated; (x, y), (x1, y1)(x2, y2), (x2, y1), (x1, y2) are the coordinate values ​​of the pixels respectively.

[0014] In some executable embodiments, the using a preset artificial intelligence model to perform multimodal content analysis on the image to be processed includes:

[0015] The large language model artificial intelligence Llama is used to perform multimodal content analysis on the image to be processed, obtain image features, embed the image features into the latent space, and determine the corresponding relationship between the image to be processed and the input text instruction through the attention mechanism; wherein, the attention mechanism calculation formula is as follows:

[0016]

[0017] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key, K T represents the transpose of the matrix K.

[0018] In some executable embodiments, the latent space is characterized by the following formula:

[0019] z=Encoder(x)+∈

[0020] Where D represents the discriminator, G represents the generator, x represents the real image, and z represents the latent space input; Pdata(x) is the probability density function representing the real data, that is, the probability at a point x in a given data space; p(z) is the noise distribution, which is a uniform or normal distribution.

[0021] In some executable embodiments, generating the first image by diffusion sampling includes:

[0022] Based on the image restoration model, the direction of image generation is determined, and diffusion sampling of preset steps is performed to gradually generate the first image, wherein the diffusion sampling of each step is implemented by the following formula:

[0023]

[0024] Among them, z t is the latent space representation of the current sampling step, α t is the noise weight, and z0 is the initial latent space representation.

[0025] In some executable embodiments, after generating the first image, the method further includes:

[0026] The latent space image is decoded back to the pixel space block by block through the variational autoencoder VAE to restore the image with optimized resolution; the decoding process uses the following back propagation formula:

[0027] x=Decoder(z)

[0028] Among them, z is the latent space representation and x is the decoded pixel space image.

[0029] In some executable embodiments, the adversarial training of the generator and the discriminator based on the generative adversarial network performs detail restoration of portrait features to obtain an output image, including:

[0030] Through the discriminator, the human features of the generator's image are repaired based on the adversarial loss to obtain the output image. The adversarial loss is shown as follows:

[0031]

[0032] Where D is the discriminator, G is the generator, x is the real image, and z is the latent space input. Pdata(x) is a function that represents the probability density of the real data, that is, the probability at a point x in a given data space. p(z) is the noise distribution, which is usually assumed to be uniform or normal distribution, and the generator generates fake images based on the noise.

[0033] According to a second aspect of the present application, there is provided a quality enhancement device including clothing image data, comprising:

[0034] An analysis unit, used for performing multimodal content analysis on the image to be processed using a preset artificial intelligence model, so as to parse the information of the person features and clothing in the image and embed them into a latent space;

[0035] The image encoding unit is used to call the image restoration model, perform diffusion processing on the image in the latent space based on the prior knowledge of the statistical characteristics of the image, extract the structural information and texture details in the image, and encode the image in blocks;

[0036] A sampling unit, configured to generate a first image by diffusion sampling based on the image restoration model and taking the image representation in the latent space of the image restoration model as a starting point;

[0037] The image decoding unit is used to detect the portrait features in the first image through multi-scale feature fusion and anchor point detection, and to restore the details of the portrait features based on the adversarial training of the generator and the discriminator of the generative adversarial network to obtain an output image.

[0038] In some executable embodiments, the device further includes:

[0039] The interpolation unit is used to perform bilinear interpolation scaling on the input image to be processed to improve the resolution of the image to be processed; wherein the formula of bilinear interpolation is:

[0040]

[0041] Among them, I(x1, y1), I(x2, y2), I(x2, y1), I(x1, y2) are the pixel values ​​of the adjacent pixels of the pixel to be bilinearly interpolated, and I(x, y) is the pixel value of the pixel to be bilinearly interpolated; (x, y), (x1, y1)(x2, y2), (x2, y1), (x1, y2) are the coordinate values ​​of the pixels respectively.

[0042] In some executable embodiments, the analysis unit is further used for:

[0043] The large language model artificial intelligence Llama is used to perform multimodal content analysis on the image to be processed, obtain image features, embed the image features into the latent space, and determine the corresponding relationship between the image to be processed and the input text instruction through the attention mechanism; wherein, the attention mechanism calculation formula is as follows:

[0044]

[0045] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key, K T represents the transpose of the matrix K.

[0046] According to a third aspect of the present application, an electronic device is provided, including:

[0047] at least one processor; and

[0048] a memory communicatively connected to the at least one processor; wherein,

[0049] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the quality enhancement method containing clothing image data.

[0050] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium is provided, and when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the steps of the quality enhancement method containing clothing image data.

[0051] The quality enhancement method and device, electronic device, and storage medium of clothing image data included in this application use the original image as the image parameter input. This application supports the input of text instructions, that is, through a paragraph of text, the requirements for image processing are set to guide the Llama model to analyze the original image. Through the technical solution of this application, the detailed information corresponding to the original image is obtained. Through the collaborative operation of the above multiple algorithms, high-quality clothing image generation and optimization are achieved, providing more expressive high-quality data for model training in the field of artificial intelligence generated content. This application can be applied to the situation of image refinement processing to achieve deep optimization of image data.

[0052] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0054] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0055] Figure 1 A schematic diagram of a flow chart of a method for enhancing quality of clothing image data according to an embodiment of the present application is shown;

[0056] Figure 2 A schematic diagram of the composition structure of a quality enhancement device for clothing image data according to an embodiment of the present application is shown;

[0057] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0059] Figure 1 FIG. 1 is a flow chart showing a method for enhancing the quality of clothing image data according to an embodiment of the present application. Figure 1 As shown, the quality enhancement method for clothing image data in the embodiment of the present application includes the following processing steps:

[0060] Step 101, use a preset artificial intelligence model to perform multimodal content analysis on the image to be processed to parse the information of the person features and clothing in the image and embed them into the latent space.

[0061] In an embodiment of the present application, the artificial intelligence model can be a large language model meta artificial intelligence (Large Language Model Meta AI), specifically, the model can be a Llama-3.2-Vision model. The embodiment of the present application uses the Llama-3.2-Vision model to perform image content analysis and key information extraction on the processed image, and improves the expression of the clothing details and materials in the processed image by the generation model through precise description, thereby improving the quality of the processed image.

[0062] In the embodiments of the present application, the training data of the Llama model comes from a public data set, thereby ensuring the compatibility and reproducibility of the Llama model in image processing. The scale of its pre-training data set is relatively sufficient, and the data scale can continue to increase with the continuous iteration of the training model. Based on this, the Llama model has demonstrated excellent performance in the field of natural language processing and is suitable for language research and commercial applications.

[0063] Specifically, the large language model artificial intelligence Llama-3.2-Vision model is used to perform multimodal content analysis on the image to be processed, thereby obtaining image features, embedding the image features into the latent space, and determining the correspondence between the image to be processed and the input text instructions through the attention mechanism. The attention mechanism calculation formula is as follows:

[0064]

[0065] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key, K T represents the transpose of the matrix K.

[0066] In the embodiment of the present application, the architecture of the Llama-3.2-Vision model adopts pre-normalization, RMSNorm normalization function, SwiGLU activation function and rotation position embedding (RoPE), etc., which places the first layer of normalization processing before the multi-head attention layer, and the second layer of normalization is moved before the fully connected layer. The processing of residual connection is also moved after the multi-head attention layer and the fully connected layer, thereby reducing the gradient disappearance problem in the image data and improving the training stability. Its normalization process uses the RMSNorm function, so that the normalization processing part depends on the mean, and normalization is performed by calculating the RMS value of the feature vector of the image, thereby improving the processing efficiency of image data processing. The function supports the setting of dynamic scaling factors and offset parameters, and improves the flexibility of image data processing.

[0067] In the embodiment of the present application, the image to be processed is input as an image parameter into the Llama-3.2-Vision model for analysis. At the same time, a text instruction needs to be input to guide the Llama model to analyze the image to be processed. In the embodiment of the present application, as an example, the text instruction input can be: "You are an image analysis assistant in the clothing field. You need to analyze the image I input and give me feedback in the form of words. The main information analyzed includes: the gender and age of the person (if there is a person), as detailed clothing information as possible, which must include the style, color and material of the clothing, and a simple description of the background information. Let me give you an example answer: a man, 30 years old, white T-shirt, pure cotton, round neck, panda pattern, black jeans, shorts, loose, straight, coarse fiber cotton cloth, white sneakers, indoor background, yellow sofa, teacup". By combining the Llama model with the analysis of the image to be processed and the text instruction, the detailed information corresponding to the image to be processed can be obtained. In an embodiment of the present application, the text instructions can be any text requirements, and the image data processing requirements can be performed in text form. The Llama model can recognize natural language and generate corresponding image processing instructions to better implement corresponding processing of the image according to the instructions to meet the image output requirements.

[0068] In the embodiment of the present application, in order to make the processing of image data more accurate, the image to be processed is input into Llam a Before the model performs multimodal content analysis, bilinear interpolation scaling is performed on the input image to be processed to improve the resolution of the image to be processed; wherein the formula for bilinear interpolation is:

[0069]

[0070] Among them, I(x1, y1), I(x2, y2), I(x2, y1), I(x1, y2) are the pixel values ​​of the adjacent pixels of the pixel to be bilinearly interpolated, I(x, y) is the pixel value of the pixel to be bilinearly interpolated; (x, y), (x1, y1)(x2, y2), (x2, y1), (x1, y2) are the coordinate values ​​of the pixels respectively. The embodiment of the present application further makes the image to be processed clearer by improving the resolution of the image to be processed.

[0071] The latent space of the embodiment of the present application is characterized by the following formula:

[0072] z=Encoder(x)+∈

[0073] Among them, z is the latent space representation, x is the image to be processed, ∈~N(0,1), represents additive noise.

[0074] Latent space is a low-dimensional space where each image corresponds to a point. In this space, similar points represent images with similar content or style. Through the latent space model, image processing is more efficient and requires less computing power and graphics card performance.

[0075] Step 102, calling the image restoration model, based on the prior knowledge of the statistical characteristics of the image, performing diffusion processing on the image in the latent space, extracting the structural information and texture details in the image, and performing block encoding on the image.

[0076] In an embodiment of the present application, the SUPIR (Super-Resolution Image Prior) model is loaded, and its VAE (Variational Autoencoder) encoder is used to perform block encoding on the image in the latent space. After block encoding, the image is converted into 512x512 blocks and input into the latent space, reducing the demand for processing resources. The SUPIR (Super-Resolution Image Prior) model is an image restoration model based on deep learning. Its core idea is to use diffusion generative prior to guide the image restoration process. Diffusion generative prior is a priori knowledge based on the statistical characteristics of the image. By performing diffusion processing on the image, the structural information and texture details in the image can be extracted, providing strong support for subsequent image restoration. The SUPIR model is based on the latent space training of the diffusion model SDXL and is an adapter model of the SDXL model.

[0077] Stable Diffusion XL (SDXL) is a text generation model based on deep learning, which aims to solve the problems of computational efficiency and memory consumption in large-scale text generation tasks. SDXL achieves the ability to efficiently generate high-quality text with limited computing resources by introducing a series of optimization techniques such as gradient checkpointing and text encoder training.

[0078] Gradient checkpointing is a technique used to reduce memory consumption of deep learning models. During the training process, the model usually needs to store a large number of intermediate variables to facilitate backpropagation to calculate gradients. Gradient checkpointing technology significantly reduces memory usage by calculating and storing only part of the information of intermediate variables when needed. In this way, SDXL can handle larger-scale text generation tasks within limited memory space. SDXL uses two text encoders to process input text. One encoder is used to extract semantic information from the text, and the other encoder is used to generate the target text. This design enables SDXL to better understand the semantic content of the input text and generate output text that is more in line with the context.

[0079] Step 103 : Based on the image restoration model, taking the image representation in the latent space of the image restoration model as a starting point, and generating a first image through diffusion sampling.

[0080] Specifically, the direction of image generation is determined based on the image restoration model, and diffusion sampling of preset steps is performed to gradually generate the first image, wherein the diffusion sampling of each step is implemented by the following formula:

[0081]

[0082] In the embodiment of the present application, the SUPIR model is used to take the image representation z in the latent space as the starting point, and the first image is generated by diffusion sampling. The diffusion sampling formula of the SUPIR model at each step is as shown in the above formula, where z t is the latent space representation of the current sampling step, α t is the noise weight, and is a value set based on experience. The aforementioned text instructions are used as prompt inputs of the SUPIR model to guide the direction of image generation. The SUPIR model can perform 50 steps of diffusion sampling to gradually generate high-quality images.

[0083] The embodiment of the present application does not adopt the super-resolution algorithm of the traditional GAN ​​network, but instead uses an adapter model trained based on the SDXL model to better improve the image quality.

[0084] The latent space image is decoded back to the pixel space block by block through the variational autoencoder VAE to restore the image with optimized resolution; the decoding process uses the following back propagation formula:

[0085] x=Decoder(z)

[0086] Among them, z is the latent space representation and x is the decoded pixel space image.

[0087] The VAE decoder decodes the generated latent space image back to the pixel space block by block, and the VAE of the SUPIR model performs block decoding at a resolution of 512*512, thereby restoring a high-quality image with optimized resolution.

[0088] Step 104: detect the portrait features in the first image through multi-scale feature fusion and anchor point detection, and perform detail restoration of the portrait features based on adversarial training of the generator and the discriminator of the generative adversarial network to obtain an output image.

[0089] In the embodiment of the present application, the portrait feature mainly refers to the face. The RetinaFace model can be used to detect the facial features in the first image. The model structure of RetinaFace adopts the depthwise separable convolution technology to reduce the number of parameters. And the feature pyramid network (FPN) is used to extract features of different scales and improve the accuracy of detection. At the same time, the SSH structure and the Head structure are selected to improve the detection ability of small targets, and the key point detection function is added, so that the model can predict the face frame and the key points of the face at the same time.

[0090] The embodiment of the present application is based on the ResNet and FPN (Feature Pyramid Network) architecture, and implements efficient face detection through multi-scale feature fusion and anchor point detection. Its loss function includes classification loss and bounding box regression loss, which is expressed as:

[0091] L=L cls +λL box

[0092] Among them, L cls is the classification loss function, L box is the bounding box regression loss function, and λ is the weight parameter, which can be set empirically. The face in the portrait in the image is cut out through the RetinaFace model, and the coordinates of the face are obtained.

[0093] In the embodiment of the present application, after acquiring a face image, the GFPGAN model is used to perform adversarial training of the generator and the discriminator based on the GAN network to restore the face details. The goal of the generator is to maximize the quality of reconstruction, while the discriminator optimizes the output of the generator through an adversarial mechanism. The adversarial loss is expressed as:

[0094]

[0095] Where D is the discriminator, G is the generator, x is the real image, z is the latent space input; Pdata(x) is a function that represents the probability density of the real data, that is, the probability at a point x in a given data space. p(z) is the noise distribution, which is usually assumed to be uniform or normal distribution. The generator generates a forged image based on the noise. Through the GFPGAN model, the blurred, corrupted and problematic faces can be repaired to obtain the final output image.

[0096] The embodiment of the present application performs image optimization under the premise of maintaining the original resolution of the image to be processed, without excessive enlargement and reduction, thus avoiding secondary loss of image quality.

[0097] Figure 2FIG. 1 shows a schematic diagram of the composition structure of a quality enhancement device for clothing image data according to an embodiment of the present application. Figure 2 As shown, the quality enhancement device containing clothing image data in the embodiment of the present application includes:

[0098] An analysis unit 20 is used to perform multimodal content analysis on the image to be processed using a preset artificial intelligence model to analyze the information of the person features and clothing in the image and embed them into a latent space;

[0099] The image encoding unit 21 is used to call the image restoration model, perform diffusion processing on the image in the latent space based on the prior knowledge of the statistical characteristics of the image, extract the structural information and texture details in the image, and perform block encoding on the image;

[0100] A sampling unit 22, configured to generate a first image by diffusion sampling based on the image restoration model and taking the image representation in the latent space of the image restoration model as a starting point;

[0101] The image decoding unit 23 is used to detect the portrait features in the first image through multi-scale feature fusion and anchor point detection, and to restore the details of the portrait features based on the adversarial training of the generator and the discriminator of the generative adversarial network to obtain an output image.

[0102] exist Figure 2 Based on the quality enhancement device containing clothing image data shown, the device also includes:

[0103] Interpolation Unit( Figure 2 ), used to perform bilinear interpolation scaling on the input image to be processed to improve the resolution of the image to be processed; wherein the formula for bilinear interpolation is:

[0104]

[0105] Among them, I(x1, y1), I(x2, y2), I(x2, y1), I(x1, y2) are the pixel values ​​of the adjacent pixels of the pixel to be bilinearly interpolated, and I(x, y) is the pixel value of the pixel to be bilinearly interpolated; (x, y), (x1, y1)(x2, y2), (x2, y1), (x1, y2) are the coordinate values ​​of the pixels respectively.

[0106] In some executable embodiments, the analysis unit 20 is further configured to:

[0107] Using the large language model artificial intelligence Llam aPerform multimodal content analysis on the image to be processed, obtain image features, embed the image features into the latent space, and determine the correspondence between the image to be processed and the input text instruction through the attention mechanism; wherein the attention mechanism calculation formula is as follows:

[0108]

[0109] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key, K T represents the transpose of the matrix K.

[0110] The latent space of the embodiment of the present application is characterized by the following formula:

[0111] z=Encoder(x)+∈

[0112] Wherein, z is the latent space representation, x is the image to be processed, ∈~N(0,1), represents additive noise.

[0113] The sampling unit 22 of the embodiment of the present application is also used for:

[0114] Based on the image restoration model, the direction of image generation is determined, and diffusion sampling of preset steps is performed to gradually generate the first image, wherein the diffusion sampling of each step is implemented by the following formula:

[0115]

[0116] Among them, z t is the latent space representation of the current sampling step, α t is the noise weight, and z0 is the initial latent space representation.

[0117] exist Figure 2 Based on the quality enhancement device containing clothing image data shown, the device also includes:

[0118] Recovery Unit( Figure 2 ), which is used to decode the latent space image back to the pixel space block by block through the variational autoencoder VAE to restore the image with optimized resolution; wherein the decoding process uses the following back propagation formula:

[0119] x=Decoder(z)

[0120] Among them, z is the latent space representation and x is the decoded pixel space image.

[0121] The image decoding unit 23 is further used for:

[0122] Through the discriminator, the human features of the generator's image are repaired based on the adversarial loss to obtain the output image. The adversarial loss is shown as follows:

[0123]

[0124] Where D is the discriminator, G is the generator, x is the real image, z is the latent space input, Pdata(x) is a function that represents the probability density of the real data, that is, the probability at a point x in the given data space. p(z) is the noise distribution, which is usually assumed to be uniform or normal distribution, and the generator generates fake images based on the noise.

[0125] In an exemplary embodiment, the aforementioned units and the like may be implemented by one or more central processing units (CPU), graphics processing units (GPU), application specific integrated circuits (ASIC), DSPs, programmable logic devices (PLD), complex programmable logic devices (CPLD), field programmable gate arrays (FPGA), general-purpose processors, controllers, microcontrollers (MCU), microprocessors, or other electronic components.

[0126] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0127] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0128] Figure 3 FIG. 8 is a schematic block diagram of an example network element 800 that can be used to implement an embodiment of the present application. Figure 3 As shown, the network element 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the network element 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0129] Multiple components in the network element 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a data processing transceiver, etc. The communication unit 809 allows the network element 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a quality enhancement method containing clothing image data. For example, in some embodiments, the quality enhancement method containing clothing image data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the network element 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the quality enhancement method containing clothing image data described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the quality enhancement method involving clothing image data in any other suitable manner (eg, by means of firmware).

[0131] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0133] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0137] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0138] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0139] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A quality enhancement method for clothing image data, characterized in that: The method comprises: Use the preset artificial intelligence model to perform multimodal content analysis on the processed image to analyze the information of the person features and clothing in the image and embed them into the latent space; Call the image restoration model, based on the prior knowledge of the statistical characteristics of the image, perform diffusion processing on the image in the latent space, extract the structural information and texture details in the image, and encode the image in blocks; Based on the image restoration model, taking the image representation in the latent space of the image restoration model as a starting point, and generating a first image through diffusion sampling; The portrait features in the first image are detected through multi-scale feature fusion and anchor point detection, and the details of the portrait features are restored based on the adversarial training of the generator and the discriminator of the generative adversarial network to obtain an output image.

2. The quality enhancement method for clothing image data according to claim 1, characterized in that: Before performing multimodal content analysis on the image to be processed, the method further includes: The input image to be processed is subjected to bilinear interpolation scaling to improve the resolution of the image to be processed; wherein the formula of bilinear interpolation is: Among them, I(x1, y1), I(x2, y2), I(x2, y1), I(x1, y2) are the pixel values ​​of the adjacent pixels of the pixel to be bilinearly interpolated, and I(x, y) is the pixel value of the pixel to be bilinearly interpolated; (x, y), (x1, y1)(x2, y2), (x2, y1), (x1, y2) are the coordinate values ​​of the pixels respectively.

3. The quality enhancement method for clothing image data according to claim 1, characterized in that: The multimodal content analysis of the image to be processed using a preset artificial intelligence model includes: The large language model artificial intelligence Llama is used to perform multimodal content analysis on the image to be processed, obtain image features, embed the image features into the latent space, and determine the corresponding relationship between the image to be processed and the input text instruction through the attention mechanism; wherein, the attention mechanism calculation formula is as follows: Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key, K T represents the transpose of the matrix K.

4. The quality enhancement method for clothing image data according to claim 1, characterized in that: The latent space is characterized by the following formula: z=Encoder(x)+∈ Wherein, z is the latent space representation, x is the image to be processed, ∈~N(0,1), represents additive noise.

5. The quality enhancement method for clothing image data according to claim 4, characterized in that: The step of generating a first image by diffusion sampling comprises: Based on the image restoration model, the direction of image generation is determined, and diffusion sampling of preset steps is performed to gradually generate the first image, wherein the diffusion sampling of each step is implemented by the following formula: Among them, z t is the latent space representation of the current sampling step, α t is the noise weight, and z0 is the initial latent space representation.

6. The quality enhancement method for clothing image data according to claim 5, characterized in that: After generating the first image, the method further includes: The latent space image is decoded back to the pixel space block by block through the variational autoencoder VAE to restore the image with optimized resolution; the decoding process uses the following back propagation formula: x=Decoder(z) Among them, z is the latent space representation and x is the decoded pixel space image.

7. The quality enhancement method for clothing image data according to claim 1, characterized in that: The adversarial training of the generator and the discriminator based on the generative adversarial network is used to restore the details of the portrait features to obtain an output image, including: Through the discriminator, the human features of the generator's image are repaired based on the adversarial loss to obtain the output image. The adversarial loss is shown as follows: Where D represents the discriminator, G represents the generator, x represents the real image, and z represents the latent space input; Pdata(x) is the probability density function representing the real data, that is, the probability at a point x in a given data space; p(z) is the noise distribution, which is a uniform or normal distribution.

8. A quality enhancement device comprising clothing image data, characterized in that: The device comprises: An analysis unit, used for performing multimodal content analysis on the image to be processed using a preset artificial intelligence model, so as to parse the information of the person features and clothing in the image and embed them into a latent space; The image encoding unit is used to call the image restoration model, perform diffusion processing on the image in the latent space based on the prior knowledge of the statistical characteristics of the image, extract the structural information and texture details in the image, and encode the image in blocks; A sampling unit, configured to generate a first image by diffusion sampling based on the image restoration model and taking the image representation in the latent space of the image restoration model as a starting point; The image decoding unit is used to detect the portrait features in the first image through multi-scale feature fusion and anchor point detection, and to restore the details of the portrait features based on the adversarial training of the generator and the discriminator of the generative adversarial network to obtain an output image.

9. The quality enhancement device containing clothing image data according to claim 8, characterized in that: The device also includes: The interpolation unit is used to perform bilinear interpolation scaling on the input image to be processed to improve the resolution of the image to be processed; wherein the formula of bilinear interpolation is: Among them, I(x1, y1), I(x2, y2), I(x2, y1), I(x1, y2) are the pixel values ​​of the adjacent pixels of the pixel to be bilinearly interpolated, and I(x, y) is the pixel value of the pixel to be bilinearly interpolated; (x, y), (x1, y1)(x2, y2), (x2, y1), (x1, y2) are the coordinate values ​​of the pixels respectively.

10. The quality enhancement device containing clothing image data according to claim 8, characterized in that: The analysis unit is further used for: The large language model artificial intelligence Llama is used to perform multimodal content analysis on the image to be processed, obtain image features, embed the image features into the latent space, and determine the corresponding relationship between the image to be processed and the input text instruction through the attention mechanism; wherein, the attention mechanism calculation formula is as follows: Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key, K T represents the transpose of the matrix K.

Images