Image rain removal model training method, device, program product, and storage medium

Abstract

The application provides a kind of image rain removal model training method, equipment, storage medium and program product;Including: obtaining sample image;Sample image is rain image;Using rain streak feature extraction module, the feature of sample image is extracted, and rain streak feature is obtained;Rain streak feature extraction module includes content quantum multiscale residual network, and quantum multiscale residual network at least contains quantum dilated convolution layer with different expansion rate;Using quantum generator in initial quantum generative adversarial network, rain streak feature and sample image are processed to obtain rain removal image;Using discriminator in initial quantum generative adversarial network, rain removal image and real rain-free image are processed to obtain discrimination result;Based on discrimination result, and the similarity between rain removal image and rain-free image, target loss function is constructed;Based on target loss function, the parameters of quantum generator and discriminator are updated to obtain target quantum generative adversarial network.

Description

Technical Field

[0001] This application relates to computer vision and quantum computing technologies, and in particular to a method, device, program product, and storage medium for training an image deraining model. Background Technology

[0002] Images acquired during rainy weather often suffer from background information loss due to rain streaks, severely impacting the reliability of outdoor vision systems such as autonomous driving and security monitoring. Currently, most image deraining methods rely on classical optimization models, utilizing prior image information to separate the rain layer from the background, but with limited effectiveness. With the development of deep learning, methods based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs) have significantly improved deraining performance. However, classical CNNs still have limitations when dealing with complex and varied real rain streaks, namely limited receptive fields and insufficient feature extraction capabilities, easily leading to problems such as loss of texture details and residual fine rain streaks in the restored image, and incurring significant computational costs. Meanwhile, quantum computing, as an emerging computing paradigm, offers new avenues for accelerating and enhancing classical algorithms through its superposition and entanglement properties. Quantum GANs have already demonstrated potential in image tasks. Therefore, how to combine the representational advantages of quantum computing with image deraining tasks to design novel deraining methods that can more effectively extract multi-scale rain streak features and restore high-quality background details has become a pressing technical problem in this field. Summary of the Invention

[0003] This application provides an image deraining model training method, device, program product, and storage medium, which can obtain a target quantum generative adversarial network that can achieve higher quality and fewer artifacts in image deraining.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides an image deraining model training method, the method comprising: Acquire a sample image; wherein the sample image is a rainy image; The sample image is processed by a rain pattern feature extraction module to obtain rain pattern features; wherein, the rain pattern feature extraction module includes a quantum multi-scale residual network, and the quantum multi-scale residual network contains at least quantum dilated convolutional layers with different dilation rates; The rain pattern features and the sample image are processed using the quantum generator in the initial quantum generative adversarial network to obtain a de-rained image; The discriminator in the initial quantum generative adversarial network is used to process the rain-removed image and the real rain-free image to obtain a discrimination result; wherein, the discrimination result represents the probability of classifying the rain-removed image as the rain-free image; Based on the discrimination results and the similarity between the rain-removed image and the rain-free image, a target loss function is constructed; Based on the target loss function, the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network are updated to obtain the target quantum generative adversarial network.

[0005] In the above scheme, the step of using a rain pattern feature extraction module to extract features from the sample image to obtain rain pattern features includes: For each residual block in the quantum multi-scale residual network, a quantum dilated convolutional layer with a different dilation rate is used to extract features from the sample image in parallel to obtain multi-scale image features. The multi-scale image features are spliced ​​and fused along the channel dimension to obtain the rain pattern features.

[0006] In the above scheme, the step of using quantum dilated convolutional layers with different dilation rates to perform parallel feature extraction on the sample image to obtain multi-scale image features includes: For each quantum dilated convolutional layer, a target encoding algorithm is used to encode the sample image to obtain the initial quantum state; The initial quantum state is subjected to a unitary transformation using a parameterized quantum circuit to obtain the transformed quantum state; The transformed quantum state is measured to obtain the expected value of the observed quantity, and the expected value is determined to be the multi-scale image feature.

[0007] In the above scheme, the step of using the quantum generator in the initial quantum generative adversarial network to process the rain pattern features and the sample image to obtain the derained image includes: The rain pattern features and the image features of the sample image are concatenated along the channel dimension to obtain the concatenated features; The splicing features are encoded using a target encoding algorithm to obtain the input quantum state; The input quantum state is subjected to a unitary transformation using multiple quantum convolutional layers in the quantum generator to obtain the output quantum state. The output quantum state is measured to obtain the rain removal image.

[0008] In the above scheme, the step of using multiple quantum convolutional layers in the quantum generator to perform a unitary transformation on the input quantum state to obtain the output quantum state includes: For each of the quantum convolutional layers, the input quantum state is transformed by a set of unitary transformation quantum gates to obtain the output quantum state; wherein each set of unitary transformation quantum gates consists of alternating rotation layers and entanglement layers, the rotation layers consist of single-qubit rotation gates acting on each qubit, and the entanglement layers consist of multi-qubit entanglement gates that realize entanglement between qubits.

[0009] In the above scheme, the step of constructing a target loss function based on the discrimination result and the similarity between the rain-removed image and the rain-free image includes: Based on the brightness, contrast, and structural similarity between the rain-removed image and the rain-free image, a structural similarity loss function is constructed. Based on the discrimination results, an adversarial loss function is constructed; Based on the structural similarity loss function and the adversarial loss function, the target loss function is constructed.

[0010] In the above scheme, acquiring the sample image includes: Acquire an initial rainy image and downsample the initial rainy image to obtain the image to be processed; The image to be processed is normalized to obtain the sample image.

[0011] This application provides an image deraining model training device, the device comprising: An acquisition unit is used to acquire a sample image; wherein the sample image is a rainy image; The feature extraction unit is used to extract features from the sample image using the rain pattern feature extraction module to obtain rain pattern features; wherein, the rain pattern feature extraction module includes a quantum multi-scale residual network, and the quantum multi-scale residual network contains at least quantum dilated convolutional layers with different dilation rates; The processing unit is used to process the rain pattern features and the sample image using a quantum generator in the initial quantum generative adversarial network to obtain a de-raining image; The processing unit is further configured to use the discriminator in the initial quantum generative adversarial network to process the rain-removed image and the real rain-free image to obtain a discrimination result; wherein, the discrimination result represents the probability of classifying the rain-removed image as the rain-free image; A construction unit is used to construct a target loss function based on the discrimination result and the similarity between the rain-removed image and the rain-free image; The generation unit is used to update the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network based on the target loss function, so as to obtain the target quantum generative adversarial network.

[0012] This application embodiment provides an image deraining model training device, the device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the image rain removal model training method provided in the embodiments of this application.

[0013] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the image rain removal model training method provided in this application when executed by a processor.

[0014] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the image rain removal model training method provided in this application.

[0015] The embodiments of this application have the following beneficial effects: By introducing a rain ripple feature extraction module using a quantum multi-scale residual network and a quantum generative adversarial network, the limitations of related rain removal methods in handling complex rain ripples are effectively overcome. First, by using quantum dilated convolutional layers with different dilation rates to extract multi-scale features in parallel, the receptive field can be expanded without increasing the number of parameters, significantly improving the ability to capture rain stripes of different sizes and directions, and alleviating the gradient vanishing problem of deep networks. Second, by using a quantum generator to perform unitary transformation and recovery on the spliced ​​features that fuse rain stripe features and original image information, the advantages of quantum entanglement in modeling complex dependencies between pixels are fully utilized, thereby better preserving the structure and texture details of the background while removing rain ripples. Furthermore, by combining structural similarity loss and adversarial loss to guide network training, the generated image is ensured to be highly consistent with the real rainless image in terms of pixel-level structure, and the visual realism of the result is improved through the adversarial game mechanism. In summary, the target quantum generative adversarial network trained by this method can achieve higher quality and fewer artifacts in image rain removal. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the image deraining model training method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the first process of the image rain removal model training method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the residual block in the image rain removal model training method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the quantum dilated convolutional layer in the image rain removal model training method provided in this application embodiment; Figure 5 This is a schematic diagram of the quantum generator in the image rain removal model training method provided in the embodiments of this application; Figure 6 This is a schematic diagram of the unitary transformation quantum gate set in the image deraining model training method provided in the embodiments of this application; Figure 7 This is a schematic diagram of the first process of the image rain removal model training method provided in the embodiments of this application; Figure 8 This is a schematic diagram of the image deraining model training device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the image rain removal model training device provided in the embodiments of this application.

[0017] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that images captured during rainy weather lack background information due to the obscuring effect of rain streaks. Rain streaks are semi-transparent and vary in size, orientation, and even appearance depending on their distance from the camera. Therefore, accurately detecting and removing rain streaks to recover rain-free images is a challenging task.

[0020] Image deraining methods in related technologies typically utilize classical optimization models to model the physical information of rain streaks. These methods, based on prior information about the image, divide the rainy image into a rain streak layer and a rain-free layer, and use low-rank appearance models, sparse coding, and Gaussian mixture models to model the rain streaks and rain-free background for deraining. Subsequently, deep learning-based image deraining algorithms have significantly improved performance. These algorithms utilize deep learning models such as multi-scale networks, recurrent neural networks, and generative adversarial networks for rain streak removal. However, due to the limitations of classical convolutional neural networks in handling complex data and feature extraction, existing image deraining algorithms perform poorly when dealing with complex real-world rainy images, easily resulting in problems such as loss of image texture details and residual fine rain streaks, and also incurring high computational costs. Quantum computing, as a new computing paradigm that can accelerate classical computing, is increasingly merging with classical artificial intelligence algorithms. Among them, quantum generative adversarial networks have shown good performance in image-related tasks, providing a possibility for improving the performance of image deraining algorithms.

[0021] Based on this, this application proposes a model training method for quantum generative adversarial networks that can be used for image deraining, referring to... Figure 1 As shown, the process includes two stages: rain pattern extraction and image restoration. In the first stage, the sample image (i.e., the rainy image) is input into the rain pattern feature extraction module. A quantum multi-scale residual network within this module mitigates gradient vanishing by using residual connections. Rain pattern features are extracted through quantum dilation convolutions at different scales. The sample image and the obtained rain pattern features are then input into the quantum generator in the initial quantum generative adversarial network (GAN). A parameterized quantum circuit composed of rotation gates and multi-qubit gates removes the rain patterns from the sample image, resulting in a de-rained image. Simultaneously, the discriminator in the initial GAN ​​distinguishes between the generated de-rained image and the real rain-free image, ensuring consistency between the generated de-rained image and the ground truth (i.e., the real rain-free image). The parameters of the quantum generator and discriminator are alternately optimized to obtain the trained target GAN.

[0022] Specifically, embodiments of this application provide a method for training an image rain removal model, referring to... Figure 2 As shown, the method applied to image deraining model training equipment can specifically include the following steps: Step 101: Obtain sample images.

[0023] The sample images are rainy images.

[0024] In this embodiment of the application, the sample image refers to the image obtained after preprocessing the initial rainy image. The initial rainy image refers to the degraded image that is visually obscured and interfered with by rain streaks when collected under rainy weather conditions. In other words, the sample image is the image obtained after preprocessing the degraded image that is visually obscured and interfered with by rain streaks when collected under rainy weather conditions.

[0025] Step 102: Use the rain pattern feature extraction module to extract features from the sample image to obtain rain pattern features.

[0026] The rain ripple feature extraction module includes a quantum multi-scale residual network, which contains at least quantum dilated convolutional layers with different dilation rates.

[0027] In this embodiment, the rain pattern feature extraction module is used to separate rain pattern information from a rainy image; the quantum multi-scale residual network can be composed of residual blocks with multiple layers, and the specific structure within each residual block is as follows: Figure 3As shown (i.e., a residual block with 4 layers), it directly passes the input to subsequent layers through "short-circuit connections," effectively alleviating the gradient vanishing problem in deep network training, allowing the network to be designed to be deeper and more expressive; the quantum dilated convolutional layer is a special type of quantum convolution operation, refer to... Figure 4 As shown, by introducing the hyperparameter of dilation rate, it expands the receptive field without increasing the number of parameters, enabling a single quantum gate operation to aggregate information from a wider area of ​​the image. It should be noted that the dilation rate determines the stride of the input pixel sampling, enabling it to capture a larger receptive field without introducing more learnable parameters. Standard convolution can be regarded as dilated convolution with a dilation rate of 1.

[0028] In this embodiment of the application, a rain pattern feature extraction module composed of multiple residual blocks is input with a rain image, targeting features such as... Figure 3 Each residual block shown is processed in parallel using multiple (e.g., 3) quantum dilation convolutional layers with different dilation rates to process the rainy image. The quantum dilation convolutional layers with small dilation rates focus on local fine rain streaks, while the quantum dilation convolutional layers with large dilation rates capture a wider range of rain streak contextual information. The multi-scale image features extracted in parallel by these quantum dilation convolutional layers are stitched together and then fused by a standard quantum convolutional layer to finally output the rain streak features. ),Right now ,in, This represents quantum dilated convolution computation, and ; This represents standard quantum convolution computation; thus, 1. through quantum dilated convolution, utilizing the superposition of quantum states, multi-scale features are efficiently extracted with fewer parameters, overcoming the limitation of the limited receptive field of classical convolution; 2. the multi-scale design can simultaneously capture rain patterns of different sizes, directions, and shapes, especially addressing the problem of fine rain pattern residue, improving the completeness of feature representation; 3. the residual structure ensures the stability of deep quantum network training, enabling the network to learn effectively; in a feasible implementation, when each residual block is a 4-layer residual block, a convolution kernel size of [missing information] can be used. Three quantum dilation convolutional layers with different dilation rates are used to process rainy images, which are then fused together by a standard quantum convolutional layer to obtain rain pattern features.

[0029] Step 103: Use the quantum generator in the initial quantum generative adversarial network to process the rain pattern features and sample images to obtain the de-rained image.

[0030] In this embodiment, the quantum generative adversarial network (QGAN) is a novel model combining quantum computing and GAN. The quantum generator, a part of the QGAN, is a parameterized quantum circuit that "generates" or "reconstructs" data based on input information. First, the obtained rain ripple features are concatenated with the image features of the sample image along the channel dimension to form an input that integrates "background information" and "rain ripple information to be removed." Then, this input is encoded into a quantum state and fed into the quantum generator, referencing... Figure 5 As shown, the quantum generator consists of multiple quantum convolutional layers, each layer passing through a set of unitary transformation quantum gates composed of rotation gates and entanglement gates (such as CNOT gates) (see reference). Figure 6 (As shown) The quantum state is transformed to gradually remove rain pattern features and restore background details. Finally, the output quantum state is measured to reconstruct the de-rained image. In this way, 1. the sample image and rain pattern features are input together, providing the generator with the complete context needed to restore the background and avoiding information loss; 2. by using a quantum entanglement gate, the generator can efficiently model the complex global dependencies between image pixels, which is crucial for restoring natural texture and structural details; 3. the parameterized quantum circuit can learn the optimal transformation from "rainy" to "no rain" through training, generating a de-rained image with high visual quality and good detail preservation.

[0031] Step 104: Use the discriminator in the initial quantum generative adversarial network to process the rain-removed image and the real rain-free image to obtain the discrimination result.

[0032] The discrimination result represents the probability of classifying a rain-free image as a rainless image.

[0033] In this embodiment, the discriminator is another part of QGAN. Specifically, it can be a classical discriminator, which consists of multiple classical linear layers. The weights are updated through the backpropagation algorithm to minimize the loss function, so that it can better distinguish between real data (i.e., rainless images) and generated data (i.e., de-rained images). The real rainless images and the de-rained images generated by the quantum generator can be input into the discriminator in the initial quantum generative adversarial network. Then, the discriminator outputs a probability value (such as the value after passing through the sigmoid function), which is usually between 0 and 1, to represent the confidence that the de-rained image is judged as a real rainless image.

[0034] Step 105: Based on the discrimination results and the similarity between the rain-removed image and the rain-free image, construct the target loss function.

[0035] In this embodiment, the target loss function is used to measure the difference between the current output of the quantum generative adversarial network (i.e., the derained image) and the ideal target (i.e., the real rainless image). It serves as a compass to guide the optimization of the initial quantum generative adversarial network parameters. The adversarial loss function can be determined based on the discrimination result. In addition, the structural similarity loss function is determined based on the similarity between the generated derained image and the real rainless image. Then, the target loss function is determined by combining the adversarial loss function and the structural similarity loss function. It should be noted that the introduction of the structural similarity loss function can effectively guide the generator to retain the structural information such as the edges and textures of the image, solving the problem that the use of adversarial loss alone may lead to texture blurring or distortion, thereby obtaining a derained image with better perceptual quality.

[0036] Step 106: Based on the target loss function, update the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network to obtain the target quantum generative adversarial network.

[0037] In this embodiment, the constructed target loss function is used to calculate the gradient of the loss with respect to the parameters of the quantum generator (parameters in the quantum gate rotation angle) and the parameters of the discriminator (parameters in the weight matrix) using the backpropagation algorithm. Subsequently, an optimizer (e.g., the Adam optimizer) can be used to iteratively update these parameters based on the gradients. The quantum generator and discriminator are optimized alternately in a "min-max game". After sufficient iterations, the parameters of the initial quantum generative adversarial network converge, resulting in a trained and optimal target quantum generative adversarial network.

[0038] The image deraining model training method provided in this application effectively overcomes the limitations of related deraining methods in handling complex rain patterns by introducing a rain pattern feature extraction module using a quantum multi-scale residual network and a quantum generative adversarial network. First, it utilizes parallel extraction of multi-scale features using quantum dilation convolutional layers with different dilation rates, which expands the receptive field without significantly increasing parameters, greatly improving the ability to capture rain patterns of different sizes and directions, and alleviating the gradient vanishing problem in deep networks. Second, it uses a quantum generator to perform unitary transformation and recovery on the spliced ​​features that fuse rain pattern features with original image information, fully utilizing the modeling advantage of quantum entanglement on complex dependencies between pixels, thus better preserving the structure and texture details of the background while removing rain patterns. Third, it combines structural similarity loss and adversarial loss to guide network training, ensuring that the generated image is highly consistent with the real rain-free image in terms of pixel-level structure, and improving the visual realism of the result through an adversarial game mechanism. In summary, the target quantum generative adversarial network trained by this method can achieve higher quality image deraining with fewer artifacts.

[0039] Based on the foregoing embodiments, this application provides yet another method for training an image rain removal model, referring to... Figure 7 As shown, the method applied to image deraining model training equipment can specifically include the following steps: Step 201: Obtain the initial rainy image and downsample the initial rainy image to obtain the image to be processed.

[0040] Step 202: Normalize the image to be processed to obtain the sample image.

[0041] In this application embodiment, the initial rainy image refers to a raw resolution rainy image directly acquired from a real scene (such as a surveillance camera or vehicle-mounted camera) or a standard dataset, and its size is usually large (e.g., (pixels); Because quantum computers are currently in the era of Noisy Intermediate-Scale Quantum (NISQ), the number of qubits and the depth of quantum circuits are limited. To reduce the requirement for the number of qubits, it is necessary to preprocess the initial rainy image (including downsampling and normalization) to obtain a sample image. Specifically, the initial rainy image can be downsampled to reduce its size (e.g., from...). Down to The image to be processed is obtained by reducing the amount of data (pixels), and then normalizing the image to obtain the sample image. It should be noted that the purpose of normalizing the image to be processed is to prevent gradient vanishing and gradient exploding. At the same time, in quantum neural networks, pixel values ​​are usually encoded as the angle parameters of quantum rotation gates. If the scale difference of the input data is too large, it will lead to extreme angle distribution of the encoded quantum state, which will cause the parameterized quantum circuit to fall into a poor optimization state in the early stage of training and be difficult to converge. Therefore, 1. Preprocessing can unify the scale of all input data and ensure that the propagation of gradients in quantum circuits and classical networks is within a reasonable numerical range; 2. Adapting quantum encoding: creates conditions for subsequently encoding classical pixel values ​​stably and consistently into the rotation angle of qubits.

[0042] Step 203: For each residual block in the quantum multi-scale residual network, use quantum dilated convolutional layers with different dilation rates to extract features from the sample image in parallel to obtain multi-scale image features.

[0043] In this embodiment, the residual block is a basic building block of the quantum multiscale residual network, and its specific structure is as follows: Figure 3As shown, it includes multiple parallel processing paths and a cross-layer connection, aiming to enable the network to learn the residual (i.e. the changing part) between the input and output, thereby alleviating the gradient vanishing problem in deep network training; parallel feature extraction refers to the simultaneous and independent processing of the input image by multiple quantum dilation convolutional layers with different dilation rates within the same residual block, rather than processing them sequentially; multi-scale image features refer to feature maps extracted by convolutional layers with different dilation rates that reflect the contextual information of different ranges of the image.

[0044] In this embodiment of the application, in each residual block, the sample image is simultaneously fed into multiple (e.g., three) quantum dilated convolutional layers, and these layers have the same kernel size (e.g., ...). However, the dilation rates differ (e.g., 1, 2, and 3 respectively). The dilation rate determines the interval between pixels processed by the convolutional kernel, thus giving each layer a different receptive field. Through this parallel structure, the network can capture multi-scale rain pattern features from local details to global context at once. In this way, 1. the parallel structure allows the network to analyze the characteristics of the image at different spatial scales simultaneously, which is crucial for detecting and modeling rain patterns with varying appearances, sizes, and orientations; 2. Quantum dilated convolution expands the receptive field by adjusting the dilation rate rather than increasing the kernel size or network depth, introducing richer contextual information while avoiding a significant increase in the number of trainable parameters, reducing model complexity and the risk of overfitting; 3. Features at different scales form a more comprehensive and complementary representation of rain patterns, laying the foundation for accurate separation of rain patterns from the background.

[0045] It should be noted that step 203 can be achieved in the following way: Step 203A1: For each quantum dilated convolutional layer, the sample image is encoded using a target encoding algorithm to obtain the initial quantum state.

[0046] In this embodiment, the initial quantum state is the encoded state of the quantum system representing the input data (i.e., the sample image), specifically referring to a superposition state of a set of qubits. Currently, various encoding methods exist, such as angle encoding and amplitude encoding. Among these, angle encoding is the most commonly used. It treats the sample image as the rotation angle of a single-qubit rotating gate and encodes it into the initial state of the qubit. Based on this, an angle encoding algorithm can be used to encode the sample image; that is, the sample image passes through an encoding layer using code operators. The initial quantum state can be represented as: ,in, It is the input vector (i.e., the sample image). It includes a single-qubit rotation gate (RY gate) and a Hadamard gate. The Hadamard gate is used to transform the initial state into a uniform superposition state, thereby increasing the complexity of the state; that is, it maps each data value of the sample image (such as the normalized pixel intensity or feature value) to the rotation angle of a single-qubit rotation gate (such as the RY gate).

[0047] Step 203A2: Perform a unitary transformation on the initial quantum state using a parameterized quantum circuit to obtain the transformed quantum state.

[0048] In this embodiment, the parameterized quantum circuit consists of a series of quantum gates (e.g., CNOT gates and single-qubit rotation gates (RY gates)) and includes multiple trainable parameters (e.g., the rotation angle of the quantum rotation gate) for extracting image features; the unitary transformation is a transformation performed by the unitary gate, where the unitary gate is the basic operation unit of quantum computing; the transformed quantum state is the state of the quantum system after processing by the parameterized quantum circuit, which contains feature information learned through training; the initial quantum state is input into the parameterized quantum circuit for unitary operation (…). ), transforming it into a new quantum state (i.e., the transformed quantum state): .

[0049] Step 203A3: Measure the transformed quantum state to obtain the expected value of the observed quantity, and determine the expected value as a multi-scale image feature.

[0050] In this embodiment, measurement is the process of extracting quantum information into classical information in quantum computing. Measurement causes the quantum state to collapse to a specific ground state, yielding a definite classical result. The observable is a Hermitian operator in quantum mechanics corresponding to a physical quantity (such as the Z component of spin). Measurement involves sampling the eigenvalues ​​of this operator. The expected value is the statistical average of the classical results obtained after multiple repeated measurements of the quantum state; it is the standard way to convert quantum information into a stable and usable classical numerical output. The transformed quantum state (… Select a local observable ( (i.e., the observable quantity) (e.g., measuring the spin in the Z direction of m qubits, where m is less than or equal to the total number of qubits in the quantum system), then performing multiple (or an infinite number of) measurements using a quantum computer or simulator, statistically analyzing these measurement results, and calculating the expected value of the observable quantity: That is, mapping the transformed quantum state to a classical vector. And this expected value is defined as the image feature output by the quantum dilated convolutional layer (i.e., multi-scale image feature).

[0051] Step 204: The multi-scale image features are spliced ​​and fused along the channel dimension to obtain rain pattern features.

[0052] In this embodiment of the application, multi-scale image features can be stitched together along the channel dimension first, and then the stitched features can be fused together to obtain rain pattern features.

[0053] Step 205: Concatenate the rain pattern features and the image features of the sample image along the channel dimension to obtain the concatenated features.

[0054] In this embodiment, the feature map of an image can be regarded as a three-dimensional tensor, whose dimensions are usually [height, width, number of channels], where the channel dimension is the third dimension. Therefore, concatenating the rain pattern feature and the image feature of the sample image along the channel dimension means connecting the rain pattern feature and the image feature of the sample image along the channel direction (the third dimension) to generate a new feature map with more channels (i.e., concatenated feature). In one feasible implementation, if the rain pattern feature has C1 channels and the image feature of the sample image has C2 channels, then the concatenated feature has C1+C2 channels.

[0055] Step 206: Encode the splicing features using a target encoding algorithm to obtain the input quantum state.

[0056] In this embodiment of the application, encoding the splicing features is similar to encoding the sample image in step 203A1, and will not be described again here.

[0057] Step 207: Use multiple quantum convolutional layers in the quantum generator to perform unitary transformation on the input quantum state to obtain the output quantum state.

[0058] In this embodiment, the quantum generator consists of n quantum convolutional layers. Each quantum convolutional layer includes multiple unitary transformation quantum gate sets for two-qubit systems, which act on adjacent qubits to achieve entanglement between the current qubit and its surrounding qubits. While retaining the important characteristics of local connectivity and weight sharing in classical convolutional layers, it improves the expressive power of quantum convolutional neural networks. That is, the quantum convolutional layer is the basic processing unit in the quantum generator, and its function is similar to that of the convolutional layer in a classical convolutional neural network, used for extracting and transforming features. The difference is that it uses quantum gates to operate on quantum states.

[0059] It should be noted that step 207 can be achieved in the following way: Step 207B: For each quantum convolutional layer, the input quantum state is transformed through a unitary transformation quantum gate set to obtain the output quantum state.

[0060] Each unitary transformation quantum gate set consists of alternating rotation layers and entanglement layers. The rotation layers consist of single-qubit rotation gates acting on each qubit, and the entanglement layers consist of multi-qubit entanglement gates that realize entanglement between qubits.

[0061] In this embodiment, the unitary transform quantum gate set is a specific combination of quantum gates that constitutes a quantum convolutional layer; it is a parameterized quantum circuit. The structure of the unitary transform quantum gate set is as follows: Figure 6 As shown, it consists of alternating rotation layers and entanglement layers. Due to the unitary property of quantum gate sets, if the rotation layers and entanglement layers operate alternately and form a sufficiently long sequence, theoretically, any unitary transformation can be performed on the initial state. The rotation layers consist of single-qubit rotation gates (RY gates) and single-qubit rotation gates (RZ gates). The entanglement layers use two-qubit logic gates (CNOT gates) arranged in various combinations. It has 15 adjustable parameters; the input quantum state passes through multiple quantum convolution layers (… After combining unitary operations, it is converted into an output quantum state.

[0062] Step 208: Measure the output quantum state to obtain the rain removal image.

[0063] In this embodiment, measurements are performed on the qubits (or a portion thereof) in the output quantum state to measure the expected value of a certain observable (e.g., the projection of each qubit onto the Z-axis). The measured values ​​(usually 0 or 1, or continuous values ​​calculated from the expected value) are then organized and reconstructed, and these values ​​are arranged into an image format (e.g., The pixel matrix is ​​used to obtain the derained image.

[0064] Step 209: Use the discriminator in the initial quantum generative adversarial network to process the rain-removed image and the real rain-free image to obtain the discrimination result.

[0065] The discrimination result represents the probability of classifying a rain-free image as a rainless image.

[0066] Step 210: Construct a structural similarity loss function based on the brightness, contrast, and structural similarity between the rain-removed image and the rain-free image.

[0067] In this embodiment, the Structure Similarity Index Measure (SSIM) is a metric used to evaluate the structural similarity between two images. It considers the structural information of the images, not just pixel-level similarity, and quantifies their similarity by comparing brightness, contrast, and structure. This makes SSIM more sensitive to capturing perceptual differences and better simulates human eye perception of images. The structure similarity loss function is expressed as follows: ,in, Indicates an image without rain. Represents a rain-removed image; and It is a constant. and , respectively, are the mean and variance of the images (i.e., the image without rain and the image without rain).

[0068] Step 211: Based on the discrimination results, construct the adversarial loss function.

[0069] In this embodiment of the application, the ability of the discriminator to identify real rainless images (i.e. ) and the discriminator's ability to recognize the generated derained images ( To construct the adversarial loss function, it can be specifically expressed as: ,in, This represents the convolution operation of the discriminator.

[0070] Step 212: Construct the target loss function based on the structural similarity loss function and the adversarial loss function.

[0071] In this embodiment, since SSIM is more sensitive to capturing perceptual differences and can better simulate human eye perception of images, a combination of structural similarity loss function and adversarial loss function is used to construct the target loss function to supervise network training; the target loss function can be specifically expressed as: ,in, This is a hyperparameter used to adjust the weights of the loss.

[0072] Step 213: Based on the target loss function, update the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network to obtain the target quantum generative adversarial network.

[0073] It should be noted that this application enhances the receptive field through quantum dilated convolution, thereby better extracting image contextual information. Based on quantum generative adversarial networks, it leverages the properties of quantum superposition and entanglement to improve network performance, effectively removing rain patterns and restoring rain-free backgrounds. Furthermore, addressing the issues of existing algorithms losing image texture details and leaving behind fine rain patterns, this application models rain pattern data using a quantum multi-scale residual network and utilizes quantum dilated convolutional layers with different receptive fields to obtain features at different scales, more effectively extracting complex rain pattern shapes.

[0074] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0075] The image deraining model training method provided in this application effectively overcomes the limitations of related deraining methods in handling complex rain patterns by introducing a rain pattern feature extraction module using a quantum multi-scale residual network and a quantum generative adversarial network. First, it utilizes parallel extraction of multi-scale features using quantum dilation convolutional layers with different dilation rates, which expands the receptive field without significantly increasing parameters, greatly improving the ability to capture rain patterns of different sizes and directions, and alleviating the gradient vanishing problem in deep networks. Second, it uses a quantum generator to perform unitary transformation and recovery on the spliced ​​features that fuse rain pattern features with original image information, fully utilizing the modeling advantage of quantum entanglement on complex dependencies between pixels, thus better preserving the structure and texture details of the background while removing rain patterns. Third, it combines structural similarity loss and adversarial loss to guide network training, ensuring that the generated image is highly consistent with the real rain-free image in terms of pixel-level structure, and improving the visual realism of the result through an adversarial game mechanism. In summary, the target quantum generative adversarial network trained by this method can achieve higher quality image deraining with fewer artifacts.

[0076] Based on the foregoing embodiments, this application provides an image deraining model training device, which can be applied to... Figure 2 and Figure 7 In the image deraining model training method provided in the corresponding embodiment, refer to Figure 8 As shown, the image deraining model training device 3 may include: an acquisition unit 31, a feature extraction unit 32, a processing unit 33, a construction unit 34, and a generation unit 35, wherein: Acquisition unit 31 is used to acquire a sample image; wherein the sample image is a rainy image; The feature extraction unit 32 is used to extract features from the sample image using the rain pattern feature extraction module to obtain rain pattern features; wherein, the rain pattern feature extraction module includes a quantum multi-scale residual network, and the quantum multi-scale residual network contains at least quantum dilated convolutional layers with different dilation rates; Processing unit 33 is used to process rain pattern features and sample images using a quantum generator in the initial quantum generative adversarial network to obtain a derained image; The processing unit 33 is also used to process the rain-removed image and the real rain-free image using the discriminator in the initial quantum generative adversarial network to obtain a discrimination result; wherein, the discrimination result represents the probability of judging the rain-removed image as a rain-free image; Construction unit 34 is used to construct the target loss function based on the discrimination result and the similarity between the rain-removed image and the rain-free image; The generation unit 35 is used to update the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network based on the target loss function, so as to obtain the target quantum generative adversarial network.

[0077] In other embodiments of this application, the feature extraction unit 32 is further configured to perform the following steps: For each residual block in the quantum multiscale residual network, a quantum dilated convolutional layer with a different dilation rate is used to extract features from the sample image in parallel to obtain multiscale image features; Multi-scale image features are spliced ​​and fused along the channel dimension to obtain rain pattern features.

[0078] In other embodiments of this application, the feature extraction unit 32 is further configured to perform the following steps: For each quantum dilation convolutional layer, a target encoding algorithm is used to encode the sample image to obtain the initial quantum state; A unitary transformation of the initial quantum state is performed using a parameterized quantum circuit to obtain the transformed quantum state; The transformed quantum state is measured to obtain the expected value of the observed quantity, and the expected value is determined to be a multi-scale image feature.

[0079] In other embodiments of this application, the processing unit 33 is further configured to perform the following steps: The rain pattern features and the image features of the sample images are concatenated along the channel dimension to obtain the concatenated features; The concatenated features are encoded using a target encoding algorithm to obtain the input quantum state; The input quantum state is transformed using multiple quantum convolutional layers in the quantum generator to obtain the output quantum state; The output quantum state is measured to obtain the rain removal image.

[0080] In other embodiments of this application, the processing unit 33 is further configured to perform the following steps: For each quantum convolutional layer, the input quantum state is transformed through a set of unitary transformation quantum gates to obtain the output quantum state. Each set of unitary transformation quantum gates consists of alternating rotation layers and entanglement layers. The rotation layers consist of single-qubit rotation gates acting on each qubit, and the entanglement layers consist of multi-qubit entanglement gates that realize entanglement between qubits.

[0081] In other embodiments of this application, the construction unit 34 is also configured to perform the following steps: Based on the brightness, contrast, and structural similarity between rain-removed and rain-free images, a structural similarity loss function is constructed. Based on the discrimination results, an adversarial loss function is constructed; A target loss function is constructed based on structural similarity loss function and adversarial loss function.

[0082] In other embodiments of this application, the acquisition unit 31 is further configured to perform the following steps: Acquire an initial rainy image and downsample it to obtain the image to be processed; The image to be processed is normalized to obtain the sample image.

[0083] It should be noted that a detailed explanation of the steps performed by each unit can be found in [reference needed]. Figure 2 and Figure 7 The image deraining model training method provided in the corresponding embodiment will not be described in detail here.

[0084] The image deraining model training device provided in this application effectively overcomes the limitations of related deraining methods in handling complex rain patterns by introducing a rain pattern feature extraction module using a quantum multi-scale residual network and a quantum generative adversarial network. First, it utilizes parallel extraction of multi-scale features using quantum dilation convolutional layers with different dilation rates, which expands the receptive field without significantly increasing parameters, greatly improving the ability to capture rain patterns of different sizes and directions, and alleviating the gradient vanishing problem in deep networks. Second, it uses a quantum generator to perform unitary transformation and recovery on the spliced ​​features that fuse rain pattern features with original image information, fully utilizing the modeling advantages of quantum entanglement on complex dependencies between pixels, thereby better preserving the structure and texture details of the background while removing rain patterns. Third, it combines structural similarity loss and adversarial loss to guide network training, ensuring that the generated image is highly consistent with the real rain-free image in terms of pixel-level structure, and improving the visual realism of the result through an adversarial game mechanism. In summary, the target quantum generative adversarial network trained by this method can achieve higher quality image deraining with fewer artifacts.

[0085] Based on the foregoing embodiments, embodiments of this application provide an image deraining model training device, which can be applied to... Figure 2 and Figure 7 In the image deraining model training method provided in the corresponding embodiment, refer to Figure 9 As shown, the image deraining model training device 4 may include: a processor 41, a memory 42, and a communication bus 43, wherein: Communication bus 43 is used to realize the communication connection between processor 41 and memory 42; Processor 41 is used to execute the image deraining model training program in memory 42 to perform the following steps: Obtain sample images; wherein, the sample images are rainy images; A rain pattern feature extraction module is used to extract features from the sample image to obtain rain pattern features; wherein, the rain pattern feature extraction module includes a quantum multi-scale residual network, and the quantum multi-scale residual network contains at least quantum dilated convolutional layers with different dilation rates; The quantum generator in the initial quantum generative adversarial network is used to process the rain pattern features and sample images to obtain de-rain images; The discriminator in the initial quantum generative adversarial network is used to process the rain-removed image and the real rain-free image to obtain the discrimination result; where the discrimination result represents the probability of classifying the rain-removed image as a rain-free image; Based on the discrimination results and the similarity between the rain-removed image and the rain-free image, a target loss function is constructed; Based on the target loss function, the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network are updated to obtain the target quantum generative adversarial network.

[0086] In other embodiments of this application, the processor 41 is used to execute the image deraining model training program in the memory 42, which uses a rain pattern feature extraction module to extract features from the sample image to obtain rain pattern features, in order to achieve the following steps: For each residual block in the quantum multiscale residual network, a quantum dilated convolutional layer with a different dilation rate is used to extract features from the sample image in parallel to obtain multiscale image features; Multi-scale image features are spliced ​​and fused along the channel dimension to obtain rain pattern features.

[0087] In other embodiments of this application, the processor 41 is used to execute the discriminator in the initial quantum generative adversarial network of the image deraining model training program in the memory 42, process the derained image and the real rainless image, and obtain a discrimination result to achieve the following steps: For each quantum dilation convolutional layer, a target encoding algorithm is used to encode the sample image to obtain the initial quantum state; A unitary transformation of the initial quantum state is performed using a parameterized quantum circuit to obtain the transformed quantum state; The transformed quantum state is measured to obtain the expected value of the observed quantity, and the expected value is determined to be a multi-scale image feature.

[0088] In other embodiments of this application, the processor 41 is used to execute the image deraining model training program in the memory 42, employing a quantum generator in the initial quantum generative adversarial network to process the rain pattern features and sample images to obtain a derained image, thereby implementing the following steps: The rain pattern features and the image features of the sample images are concatenated along the channel dimension to obtain the concatenated features; The concatenated features are encoded using a target encoding algorithm to obtain the input quantum state; The input quantum state is transformed using multiple quantum convolutional layers in the quantum generator to obtain the output quantum state; The output quantum state is measured to obtain the rain removal image.

[0089] In other embodiments of this application, the processor 41 is used to execute the image deraining model training program in the memory 42, which employs multiple quantum convolutional layers in the quantum generator to perform unitary transformation on the input quantum state to obtain the output quantum state, in order to achieve the following steps: For each quantum convolutional layer, the input quantum state is transformed through a set of unitary transformation quantum gates to obtain the output quantum state. Each set of unitary transformation quantum gates consists of alternating rotation layers and entanglement layers. The rotation layers consist of single-qubit rotation gates acting on each qubit, and the entanglement layers consist of multi-qubit entanglement gates that realize entanglement between qubits.

[0090] In other embodiments of this application, processor 41 is used to execute the image deraining model training program in memory 42 based on the discrimination result and the similarity between the derained image and the rainless image, to construct a target loss function to achieve the following steps: Based on the brightness, contrast, and structural similarity between rain-removed and rain-free images, a structural similarity loss function is constructed. Based on the discrimination results, an adversarial loss function is constructed; A target loss function is constructed based on structural similarity loss function and adversarial loss function.

[0091] In other embodiments of this application, the processor 41 is used to execute the image deraining model training program in the memory 42 to acquire sample images to implement the following steps: Acquire an initial rainy image and downsample it to obtain the image to be processed; The image to be processed is normalized to obtain the sample image.

[0092] It should be noted that a detailed description of the steps performed by the processor can be found in [reference needed]. Figure 2 and Figure 7 The image deraining model training method provided in the corresponding embodiment will not be described in detail here.

[0093] The image deraining model training device provided in this application effectively overcomes the limitations of related deraining methods in handling complex rain patterns by introducing a rain pattern feature extraction module using a quantum multi-scale residual network and a quantum generative adversarial network. First, it utilizes parallel extraction of multi-scale features using quantum dilation convolutional layers with different dilation rates, which expands the receptive field without significantly increasing parameters, greatly improving the ability to capture rain patterns of different sizes and directions, and alleviating the gradient vanishing problem in deep networks. Second, it uses a quantum generator to perform unitary transformation and recovery on the spliced ​​features that fuse rain pattern features with original image information, fully utilizing the modeling advantages of quantum entanglement on complex dependencies between pixels, thereby better preserving the structure and texture details of the background while removing rain patterns. Furthermore, it combines structural similarity loss and adversarial loss to guide network training, ensuring that the generated image is highly consistent with the real rain-free image in terms of pixel-level structure, and improving the visual realism of the result through an adversarial game mechanism. In summary, the target quantum generative adversarial network trained by this method can achieve higher quality image deraining with fewer artifacts.

[0094] Based on the foregoing embodiments, this application provides a computer program product, including a computer program, which, when executed by a processor, implements... Figure 2 and Figure 7 The corresponding embodiment provides the steps of the image deraining model training method.

[0095] Based on the foregoing embodiments, this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to achieve... Figure 2 and Figure 7 The corresponding embodiment provides the steps of the image deraining model training method.

[0096] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for training an image rain removal model, characterized in that, The method includes: Acquire a sample image; wherein the sample image is a rainy image; The sample image is processed by a rain pattern feature extraction module to obtain rain pattern features; wherein, the rain pattern feature extraction module includes a quantum multi-scale residual network, and the quantum multi-scale residual network contains at least quantum dilated convolutional layers with different dilation rates; The rain pattern features and the sample image are processed using the quantum generator in the initial quantum generative adversarial network to obtain a de-rained image; The discriminator in the initial quantum generative adversarial network is used to process the rain-removed image and the real rain-free image to obtain a discrimination result; wherein, the discrimination result represents the probability of classifying the rain-removed image as the rain-free image; Based on the discrimination results and the similarity between the rain-removed image and the rain-free image, a target loss function is constructed; Based on the target loss function, the parameters of the quantum generator and discriminator in the initial quantum generative adversarial network are updated to obtain the target quantum generative adversarial network.

2. The method according to claim 1, characterized in that, The rain pattern feature extraction module is used to extract features from the sample image to obtain rain pattern features, including: For each residual block in the quantum multi-scale residual network, a quantum dilated convolutional layer with a different dilation rate is used to extract features from the sample image in parallel to obtain multi-scale image features. The multi-scale image features are spliced ​​and fused along the channel dimension to obtain the rain pattern features.

3. The method according to claim 2, characterized in that, The method employs quantum dilated convolutional layers with different dilation rates to perform parallel feature extraction on the sample images, obtaining multi-scale image features, including: For each quantum dilated convolutional layer, a target encoding algorithm is used to encode the sample image to obtain the initial quantum state; The initial quantum state is subjected to a unitary transformation using a parameterized quantum circuit to obtain the transformed quantum state; The transformed quantum state is measured to obtain the expected value of the observed quantity, and the expected value is determined to be the multi-scale image feature.

4. The method according to claim 1, characterized in that, The process of using a quantum generator in an initial quantum generative adversarial network to process the rain pattern features and the sample image to obtain a de-rained image includes: The rain pattern features and the image features of the sample image are concatenated along the channel dimension to obtain the concatenated features; The splicing features are encoded using a target encoding algorithm to obtain the input quantum state; The input quantum state is subjected to a unitary transformation using multiple quantum convolutional layers in the quantum generator to obtain the output quantum state. The output quantum state is measured to obtain the rain removal image.

5. The method according to claim 4, characterized in that, The step of performing a unitary transformation on the input quantum state using multiple quantum convolutional layers in the quantum generator to obtain the output quantum state includes: For each of the quantum convolutional layers, the input quantum state is transformed by a set of unitary transformation quantum gates to obtain the output quantum state; wherein each set of unitary transformation quantum gates consists of alternating rotation layers and entanglement layers, the rotation layers consist of single-qubit rotation gates acting on each qubit, and the entanglement layers consist of multi-qubit entanglement gates that realize entanglement between qubits.

6. The method according to claim 1, characterized in that, Based on the discrimination result and the similarity between the rain-removed image and the rain-free image, the target loss function is constructed, including: Based on the brightness, contrast, and structural similarity between the rain-removed image and the rain-free image, a structural similarity loss function is constructed. Based on the discrimination results, an adversarial loss function is constructed; Based on the structural similarity loss function and the adversarial loss function, the target loss function is constructed.

7. The method according to claim 1, characterized in that, The acquisition of sample images includes: Acquire an initial rainy image and downsample the initial rainy image to obtain the image to be processed; The image to be processed is normalized to obtain the sample image.

8. An image deraining model training device, characterized in that, The device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that can be executed by one or more processors to implement the method of any one of claims 1 to 7.

Images