Multi-domain image enhancement method and device based on memory and decision fusion

Abstract

This invention provides a multi-domain image enhancement method and apparatus based on memory and decision fusion, relating to the field of intelligent image enhancement technology in scenarios with large distribution differences across multiple domains. The method includes: upon receiving an image to be enhanced from an unknown domain, extracting at least one image pair from a memory unit, the image pair including an original image from a known domain and an enhancement image label corresponding to the original image; invoking a domain alignment unit in a decision unit to align the image to be enhanced with each original image, obtaining aligned images corresponding to each original image; optimizing the image enhancement unit in the decision unit based on the aligned images and enhancement image labels corresponding to each original image; and performing image enhancement on the image to be enhanced based on the optimized image enhancement unit, obtaining the enhanced image corresponding to the image to be enhanced. This invention can significantly improve the image enhancement effect and cross-domain generalization ability of multi-domain images.

Description

Technical Field

[0001] This invention relates to the field of intelligent image enhancement technology in scenarios with large distribution differences across multiple domains, and particularly to a multi-domain image enhancement method and apparatus based on the fusion of memory and decision. Background Technology

[0002] In existing technologies, local window attention mechanisms or mask modeling strategies are commonly used to achieve image enhancement.

[0003] The local window attention mechanism, also known as Swin Transformer for Image Restoration (SwinIR), is based on the local window attention mechanism of the Swin Transformer. It achieves efficient cross-window information interaction and global modeling through a shifting window strategy, effectively improving image restoration performance while reducing computational complexity. The masking modeling strategy randomly covers parts of the input image, forcing the model to reconstruct the covered region only from the visible parts, thereby effectively improving the model's understanding and generalization ability of image content.

[0004] However, when using local window attention mechanisms for image enhancement, the fixed window partitioning strategy may limit the model's generalization ability and computational efficiency in scenarios with large differences in distribution across multiple domains. Similarly, when using mask modeling strategies for image enhancement, the fixed masking strategy may not be able to adequately adapt to the complexity of features in different domains, thus limiting the model's adaptability and performance.

[0005] Therefore, an effective solution is urgently needed to address at least one of the above problems. Summary of the Invention

[0006] To address at least one of the aforementioned problems, this invention provides a multi-domain image enhancement method and apparatus based on the fusion of memory and decision-making.

[0007] This invention provides a multi-domain image enhancement method based on memory and decision fusion, comprising: Upon receiving an image to be enhanced in an unknown region, at least one image pair is extracted from the memory unit, the image pair including the original image of the known region and the enhanced image label corresponding to the original image; The domain alignment unit in the decision unit is invoked to align the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images; Based on the aligned image and the enhanced image label corresponding to each of the original images, the image enhancement unit in the decision unit is optimized; Based on the optimized image enhancement unit, the image to be enhanced is enhanced to obtain the enhanced image corresponding to the image to be enhanced.

[0008] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion includes aligning the image to be enhanced with each of the original images to obtain aligned images corresponding to each of the original images, comprising: For each original image, Fourier transform, frequency domain transform, and inverse Fourier transform are performed on the original image and the image to be enhanced to obtain the aligned image corresponding to the original image.

[0009] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion is provided, wherein performing Fourier transform, frequency domain transform, and inverse Fourier transform on the original image and the image to be enhanced to obtain an aligned image corresponding to the original image includes: Fourier transform is performed on the original image and the image to be enhanced, respectively, to obtain the frequency domain features of the original image and the frequency domain features of the image to be enhanced; A distribution alignment operation is performed on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features; Perform an inverse Fourier transform on the aligned frequency domain features to obtain the aligned image corresponding to the original image.

[0010] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion is provided, wherein the frequency domain features include phase spectrum and amplitude spectrum; The step of performing a distribution alignment operation on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features includes: The central region of the amplitude spectrum of the original image is replaced with the central region of the amplitude spectrum of the image to be enhanced, while keeping the non-central region of the amplitude spectrum of the original image and the phase spectrum of the original image unchanged, to obtain the aligned frequency domain features. The size and shape of the central region are set based on task requirements.

[0011] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion is provided, wherein, upon receiving an image to be enhanced from an unknown domain, at least one image pair is extracted from a memory unit, comprising: Obtain the domain distribution parameters and the first quantity of the image to be enhanced; Based on the first quantity, determine the second quantity; Based on the domain distribution parameters of the image to be enhanced and the domain distribution parameters of each image pair stored in the memory unit, the second number of image pairs are extracted from each image pair stored in the memory unit.

[0012] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion, before aligning the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images, further includes: The image to be enhanced is preprocessed, including normalization and / or filtering.

[0013] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion is provided, the method further comprising: The aligned image corresponding to the original image and the enhanced image label of the original image are determined as a new image pair, and the new image pair is stored in the memory unit; And / or, determine the image to be enhanced and the enhanced image as a new image pair, and store the new image pair in the memory unit.

[0014] According to the present invention, a multi-domain image enhancement method based on memory and decision fusion is provided, wherein storing the new image pair into the memory unit includes: The domain distribution parameters of the original image in the new image pair and the new image pair are stored in the memory unit in a structured form.

[0015] The present invention also provides a multi-domain image enhancement device based on memory and decision fusion, comprising: The extraction module is configured to extract at least one image pair from a memory unit upon receiving an image to be enhanced in an unknown region, the image pair including the original image of the known region and the enhancement image label corresponding to the original image; The alignment module is configured to call the domain alignment unit in the decision unit to align the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images; The optimization module is configured to optimize the image enhancement unit in the decision unit based on the aligned image and the enhanced image label corresponding to each of the original images; The image enhancement module is configured to perform image enhancement on the image to be enhanced based on the optimized image enhancement unit, so as to obtain the enhanced image corresponding to the image to be enhanced.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-domain image enhancement method based on memory and decision fusion as described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-domain image enhancement method based on memory and decision fusion as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-domain image enhancement method based on memory and decision fusion as described above.

[0019] The present invention provides a multi-domain image enhancement method and apparatus based on memory and decision fusion. Upon receiving an image to be enhanced in an unknown domain, the method extracts at least one image pair from a memory unit. Each image pair includes an original image from a known domain and an enhanced image label corresponding to the original image. A domain alignment unit in a decision unit is invoked to align the image to be enhanced with each of the original images, resulting in aligned images corresponding to each original image. Based on the aligned images and the enhanced image labels corresponding to each original image, the image enhancement unit in the decision unit is optimized. Finally, based on the optimized image enhancement unit, the image to be enhanced is enhanced to obtain the enhanced image corresponding to the image to be enhanced.

[0020] This invention can receive image input from different domains. By introducing a memory unit to store existing image pairs and combining it with a domain alignment unit in the decision unit, it adaptively aligns the image to be enhanced with each original image in the frequency domain, thereby constructing labeled data for unknown new domains and reducing data labeling costs. Furthermore, based on the aligned and enhanced image labels, the image enhancement unit is optimized, ensuring cross-domain image enhancement performance and guaranteeing that the enhancement process can adapt to the specific needs of different domains. The fusion of memory and decision not only significantly improves the quality of image enhancement but also optimizes the model's computational efficiency and cross-domain adaptability, surpassing the performance of existing technologies in complex multi-domain scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts of the multi-domain image enhancement method based on memory and decision fusion provided by the present invention.

[0023] Figure 2 This is a flowchart illustrating the alignment operation provided by the present invention.

[0024] Figure 3 This is the second flowchart of the multi-domain image enhancement method based on memory and decision fusion provided by the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of the multi-domain image enhancement device based on memory and decision fusion provided by the present invention.

[0026] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] First, a brief description of the relevant content involved in this invention will be given.

[0029] The following are the drawbacks of existing image enhancement technologies.

[0030] Model training under the assumption of a single distribution leads to degraded cross-domain performance. Existing image augmentation techniques are typically based on the assumption of a single distribution, which assumes that the feature distribution of training and inference samples is consistent, thus designing a unified augmentation strategy and model architecture. However, in multi-domain scenarios, images from different domains differ significantly in resolution, texture, illumination, and noise distribution. The single distribution assumption makes it difficult for models to capture these domain-specific features. In cross-domain tasks, models often exhibit undergeneralization on images from unseen domains, such as over-enhancing certain features or ignoring domain-specific details, resulting in unstable augmentation effects.

[0031] In multi-domain scenarios, acquiring high-quality labeled data is costly and inhibits model generalization performance. High-quality labeled data is widely relied upon to improve model training effectiveness and performance in multi-domain scenarios. However, due to significant differences in image distribution, features, and semantics across different domains, acquiring high-quality labeled data covering multi-domain features faces extremely high costs and challenges, significantly restricting data-driven model training. The scarcity of labeled data leads to a lack of effective learning of feature transfer and diversity distribution between domains during model training, resulting in a significant decline in cross-domain generalization performance. Simultaneously, data imbalance and domain-specific training samples easily cause models to overfit to specific domains, further limiting their robustness and universality in complex real-world applications.

[0032] Therefore, this invention provides a multi-domain image enhancement method and apparatus based on memory and decision fusion. It can receive image input from different domains, store existing labeled data information by introducing a memory unit, and dynamically adjust the features of specific domains by combining a decision unit, thereby achieving efficient and robust image enhancement in multi-domain scenarios. The decision unit includes a domain alignment unit based on frequency domain information and an image enhancement unit. The domain alignment unit adaptively aligns labeled data with unknown domain data in the frequency domain, constructing labeled data for unknown new domains, reducing data labeling costs and ensuring cross-domain image enhancement performance, ensuring the enhancement process can adapt to the specific needs of different domains. The image enhancement unit can perform various image enhancement tasks. The fusion of memory and decision units not only significantly improves the quality of image enhancement but also optimizes the model's computational efficiency and cross-domain adaptability, surpassing the performance of existing technologies in complex multi-domain scenarios.

[0033] The following is combined Figures 1 to 5 This invention describes a multi-domain image enhancement method and apparatus based on the fusion of memory and decision-making.

[0034] Figure 1 This is one of the flowcharts illustrating the multi-domain image enhancement method based on memory and decision fusion provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Upon receiving an image to be enhanced in an unknown region, extract at least one image pair from the memory unit, the image pair including the original image of the known region and the enhancement image label corresponding to the original image; Step 102: Call the domain alignment unit in the decision unit to align the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images; Step 103: Optimize the image enhancement unit in the decision unit based on the aligned image and the enhanced image label corresponding to each original image; Step 104: Based on the optimized image enhancement unit, perform image enhancement on the image to be enhanced to obtain the enhanced image corresponding to the image to be enhanced.

[0035] The execution subject of the multi-domain image enhancement method based on memory and decision fusion provided by the present invention can be an image enhancement system, an electronic device with image enhancement function, or other implementable objects.

[0036] Specifically, the image to be enhanced refers to the image that needs image enhancement processing. The known domain refers to the domain to which the image to which the execution subject performs enhancement processing during training and procedural inference belongs; the unknown domain, also known as a new unknown domain, refers to any neighboring domain outside the known domain. For example, the known domain includes urban topography, natural landscapes, and portraits, while the unknown domain can be interior structures, clothing designs, etc. The memory unit is used to store existing labeled data (image pairs). The domain alignment unit is used to construct labeled data (aligned image and enhanced image label corresponding to the original image) that approximates the unknown domain by adaptively aligning the labeled data (the original image in the known domain of the image pair) with the unknown domain data (the image to be enhanced in the unknown domain) in the frequency domain. The image enhancement unit can be any image enhancement model.

[0037] In practical applications, the executing entity first receives the image to be enhanced.

[0038] When the domain corresponding to the image to be enhanced is a known domain, the executing entity can directly call the image enhancement unit in the decision unit to perform image enhancement on the image to be enhanced, thereby obtaining the enhanced image corresponding to the image to be enhanced.

[0039] When the domain corresponding to the image to be enhanced is unknown, the executing entity can extract at least one image pair from the memory unit; then, through the domain alignment unit, the image to be enhanced is aligned with the original images in each image pair to obtain the aligned images corresponding to each original image; further, for each original image, a sample pair is formed using the aligned image corresponding to the original image as a sample and the enhanced image label corresponding to the original image as a label; the image enhancement unit is optimized using each sample pair to obtain an optimized image enhancement unit; then, the optimized image enhancement unit is used to enhance the image to be enhanced to obtain the enhanced image corresponding to the image to be enhanced. The optimization process may include at least one iteration.

[0040] The multi-domain image enhancement method based on memory and decision fusion provided by this invention can receive image input from different domains. By introducing a memory unit to store existing image pairs and combining it with a domain alignment unit in the decision unit, it adaptively aligns the image to be enhanced with each original image in the frequency domain, thereby constructing labeled data for unknown new domains and reducing data labeling costs. Furthermore, the image enhancement unit is optimized based on the aligned image and the labels of the enhanced image, ensuring cross-domain image enhancement performance and ensuring that the enhancement process can adapt to the specific needs of different domains. The fusion of memory and decision not only significantly improves the quality of image enhancement but also optimizes the computational efficiency and cross-domain adaptability of the model, surpassing the performance of existing technologies in complex multi-domain scenarios.

[0041] In one or more optional embodiments of the present invention, aligning the image to be enhanced with each of the original images to obtain an aligned image corresponding to each of the original images includes: For each original image, Fourier transform, frequency domain transform, and inverse Fourier transform are performed on the original image and the image to be enhanced to obtain the aligned image corresponding to the original image.

[0042] In practical applications, for the original image in each extracted image pair, Fourier transform, frequency domain transform, and inverse Fourier transform can be performed on the original image and the image to be enhanced to achieve domain alignment, thereby obtaining the aligned image corresponding to the original image.

[0043] In this embodiment of the invention, compared with existing domain alignment methods, the alignment process utilizes existing information (image pairs of known domains) stored in the memory unit to achieve domain alignment through simple frequency domain transformation and numerical operations (Fourier transform and inverse Fourier transform). It does not rely on a large amount of data for complex model training and optimization, thus enabling more efficient dynamic adjustment of specific domain features and significantly improving the speed and adaptability of the system.

[0044] In one or more optional embodiments of the present invention, performing Fourier transform, frequency domain transform, and inverse Fourier transform on the original image and the image to be enhanced to obtain the aligned image corresponding to the original image includes: Fourier transform is performed on the original image and the image to be enhanced, respectively, to obtain the frequency domain features of the original image and the frequency domain features of the image to be enhanced; A distribution alignment operation is performed on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features; Perform an inverse Fourier transform on the aligned frequency domain features to obtain the aligned image corresponding to the original image.

[0045] In practical applications, the extracted image pairs can be used as (I s L I s H ) indicates that, among which, I s L This represents low-quality data, i.e., the original image before image enhancement; I s H This represents the high-quality data corresponding to the original image, i.e., the enhanced image label.

[0046] See Figure 2 , Figure 2 This is a flowchart illustrating the alignment operation provided by the present invention: First, the original image I... s L and the image to be enhanced IL Perform a Fourier transform: on the original image I extracted from the memory cell s L Image I to be enhanced in unknown areas L Perform Fourier transforms on each image to extract its frequency domain features. During the transformation process, obtain the original image I. s L Phase spectrum F S A and amplitude spectrum F S P and the image to be enhanced I L Phase spectrum F A and amplitude spectrum F P These frequency domain features (phase spectrum and amplitude spectrum) will serve as the core input for subsequent alignment operations, used for domain alignment and dynamic feature adjustment in specific domains.

[0047] See Figure 2 After obtaining the frequency domain features of the original image and the image to be enhanced, it is necessary to modify the frequency domain information to adapt to specific domain features. In order to enable the image enhancement unit to better adapt to the new unknown domain, a distribution alignment operation is performed in the frequency domain: the frequency domain features of the original image and the frequency domain features of the image to be enhanced are distributed and aligned to obtain the aligned frequency domain features, i.e., the aligned frequency domain features.

[0048] Further, see Figure 2 An inverse Fourier transform is performed on the labeled data (aligned frequency domain features) that has undergone frequency domain distribution alignment to restore the frequency domain information to the spatial domain, thereby obtaining the aligned image I corresponding to the original image. s’ L .

[0049] In this embodiment of the invention, Fourier transform, frequency domain transform, and inverse Fourier transform are performed on the original image and the image to be enhanced to generate a labeled dataset (the aligned image and the label of the enhanced image corresponding to the original image) whose distribution is closer to the new unknown domain. The labeled dataset that is closer to the new unknown domain retains the high-quality characteristics of the original labeled data (extracted image pairs), and at the same time, its distribution characteristics are aligned with the data in the unknown domain (the image to be enhanced), which helps the image enhancement unit to adapt to the new domain requirements more efficiently and improve the effect and generalization performance of cross-domain image enhancement.

[0050] In one or more optional embodiments of the present invention, the frequency domain features include a phase spectrum and an amplitude spectrum; The step of performing a distribution alignment operation on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features includes: The central region of the amplitude spectrum of the original image is replaced with the central region of the amplitude spectrum of the image to be enhanced, while keeping the non-central region of the amplitude spectrum of the original image and the phase spectrum of the original image unchanged, to obtain the aligned frequency domain features. The size and shape of the central region are set based on task requirements.

[0051] In practical applications, see Figure 2 When performing distribution alignment, the image to be enhanced in the unknown neighborhood can be used. s L amplitude spectrum F A Replace the original image with the central region I s L amplitude spectrum F S A The corresponding central region is preserved, while keeping the original values ​​of other regions unchanged, and maintaining the phase spectrum F of both. S P and F P Unaffected.

[0052] The size of the central region is a learnable parameter that can be flexibly set to a geometric shape such as a rectangle, square, or circle according to task requirements. That is, based on the needs, the system can select the most suitable size and shape of the central region, perform distribution alignment operations, and then apply the most suitable size and shape of the central region to that area. For example... Figure 2 As shown, the central area is a square.

[0053] In this embodiment of the invention, by partially replacing frequency domain information, the spectral characteristics of labeled data are made closer to the distribution characteristics of unknown domains, thereby achieving cross-domain feature adaptation.

[0054] In one or more optional embodiments of the present invention, the step of retrieving at least one image pair from the memory unit upon receiving an image to be enhanced in an unknown region includes: Obtain the domain distribution parameters and the first quantity of the image to be enhanced; Based on the first quantity, determine the second quantity; Based on the domain distribution parameters of the image to be enhanced and the domain distribution parameters of each image pair stored in the memory unit, the second number of image pairs are extracted from each image pair stored in the memory unit.

[0055] Specifically, domain distribution parameters refer to distribution parameters related to the image domain, used to describe the indicators and numerical characteristics of various distributions in the image, such as the population mean, population standard deviation, and population correlation coefficient. The first quantity refers to the number of images to be enhanced; the second quantity is the extraction quantity, that is, the number of image pairs to be extracted.

[0056] In practical applications, the domain distribution parameters of the image to be enhanced in an unknown region can be obtained, and the first number N of the input image to be enhanced can be determined.

[0057] Furthermore, a second quantity X is determined based on a first quantity N of the image to be enhanced.

[0058] For example, the second quantity X can be determined based on the following formula: X=N M; where M is a preset multiple, which can be set according to requirements.

[0059] Based on the second number and the domain distribution parameters of the images to be enhanced, the second number of image pairs can be selected from the memory unit based on the domain distribution parameters of the images to be enhanced.

[0060] For example, the similarity between the domain distribution parameters of the image to be enhanced and the domain distribution parameters of each image pair stored in the memory unit can be calculated, and the top X image pairs with the highest similarity can be extracted and processed to obtain the second number of image pairs.

[0061] For example, the similarity between the domain distribution parameters of the image to be enhanced and the domain distribution parameters of each image pair stored in the memory unit can be calculated. The top Y image pairs with the highest similarity are extracted, and XY image pairs are randomly extracted to obtain the second number of image pairs. Here, Y is less than X.

[0062] In this embodiment of the invention, by dynamically selecting an appropriate number of image pairs from known domains, it is possible to flexibly adapt to the needs of different domains and improve the efficiency and effectiveness of domain alignment and image enhancement.

[0063] In one or more optional embodiments of the present invention, before aligning the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images, the method further includes: The image to be enhanced is preprocessed, including normalization and / or filtering.

[0064] In practical applications, after receiving the image to be enhanced, data preprocessing is required: First, normalization is performed on the image to be enhanced, mapping the image pixel values ​​to a standardized range [0, 1] to eliminate feature bias caused by differences in value ranges and ensure data consistency and comparability.

[0065] And / or, after receiving the image to be enhanced, the image can be filtered to remove redundant background information and highlight the key feature areas of the foreground, thereby effectively reducing the interference of background noise on the image enhancement effect and further improving the accuracy and robustness of data preprocessing.

[0066] For example, firstly, normalization is performed on the input image to be enhanced in the unknown region, uniformly mapping the image pixel values ​​to the normalized range [0, 1]. If necessary, such as the presence of unnecessary or redundant information like background or bright spots in the image to be enhanced, filtering is required to obtain a preprocessed image to be enhanced. Further, based on the preprocessed image to be enhanced, at least one image pair is extracted from the memory unit, and subsequent steps are performed until the enhanced image corresponding to the image to be enhanced is determined.

[0067] In one or more optional embodiments of the present invention, the method further includes: The aligned image corresponding to the original image and the enhanced image label of the original image are determined as a new image pair, and the new image pair is stored in the memory unit; And / or, determine the image to be enhanced and the enhanced image as a new image pair, and store the new image pair in the memory unit.

[0068] In practical applications, the aligned image corresponding to the original image can be used as the image before image enhancement, and the enhanced image label corresponding to the original image can be used as the enhanced image corresponding to the aligned image, forming a labeled data pair, i.e., a new image pair. Alternatively, the image to be enhanced can be used as the image before image enhancement, and the enhanced image can be used as the enhanced image corresponding to the image to be enhanced, forming a labeled data pair, i.e., a new image pair. The new image pair is then stored in a memory unit.

[0069] In addition, new image pairs can be normalized before being stored in the storage memory unit.

[0070] In this way, as the application scenarios continue to expand, the data in the memory unit will be dynamically updated, including labeled data for newly added domains, so that the executing entity can gradually accumulate and enrich domain knowledge, continuously improve its adaptability and generalization ability in multi-domain scenarios, and meet the diverse needs of complex application scenarios.

[0071] In one or more optional embodiments of the present invention, storing the new image pair into the memory unit includes: The domain distribution parameters of the original image in the new image pair and the new image pair are stored in the memory unit in a structured form.

[0072] Specifically, when the aligned image corresponding to the original image and the enhanced image label of the original image are determined as a new image pair, the image before enhancement is the aligned image; when the image to be enhanced and the enhanced image are determined as a new image pair, the image before enhancement is the image to be enhanced.

[0073] In practical applications, the domain distribution parameters of the original image in a new image pair can be obtained. These parameters, along with the new image, are then stored in a structured format in the memory unit. As the application scenarios expand, the data in the memory unit will be dynamically updated, including newly added domain-specific labeled data (image pairs) and domain distribution parameters. This allows the system to gradually accumulate and enrich domain knowledge, continuously improving its adaptability and generalization capabilities across multiple domains to meet the diverse needs of complex application scenarios.

[0074] The following is combined Figure 3 The multi-domain image enhancement method based on memory and decision fusion provided by this invention will be further described.

[0075] Figure 3 This is the second flowchart of the multi-domain image enhancement method based on memory and decision fusion provided by the present invention, as shown below. Figure 3 As shown, the method includes the following steps.

[0076] First, the input image to be enhanced is preprocessed.

[0077] Then, data extraction is performed on the memory cell: based on the preprocessed image to be enhanced, X image pairs of known neighborhoods are extracted from the memory cell, where the value of X is dynamically adjusted according to the data of the image to be enhanced in the unknown neighborhood, rather than being a fixed size.

[0078] Next, Fourier transform is performed: Fourier transform is performed on the original data and the image to be enhanced in the image pair selected from the memory cell to extract their frequency domain features, which include amplitude spectrum and phase spectrum.

[0079] Furthermore, frequency domain distribution alignment is performed: the center region of the amplitude spectrum of the image to be enhanced is used to replace the corresponding center region of the amplitude spectrum of the original data, while keeping the original values ​​of other regions unchanged and keeping the phase spectrum of both unaffected, thus obtaining the aligned frequency domain features.

[0080] Next, an inverse Fourier transform is performed: the labeled data (aligned frequency domain features) that has undergone frequency domain distribution alignment is subjected to an inverse Fourier transform to restore the frequency domain information to the spatial domain, thereby obtaining the aligned image.

[0081] Then, based on the aligned images and enhanced image labels corresponding to each original image, the image enhancement units in the decision unit are fused and optimized, and based on the optimized image enhancement units, the image to be enhanced is enhanced to obtain the enhanced image corresponding to the image to be enhanced.

[0082] In addition, the image to be enhanced and the enhanced image are treated as new image pairs, and the aligned image and the enhanced image label are treated as new image pairs. After normalization, they are stored in the memory unit in a structured form.

[0083] It is important to note that the Fourier transform and inverse transform of the data are not performed during the data preparation phase before training, but are dynamically executed within the process of the image enhancement model's inference in new, unknown domains. During inference, the data after the aforementioned frequency domain transformation is used for real-time fine-tuning to guide the image enhancement model in adjusting its parameters and feature extraction strategies, thereby achieving better generalization ability and enhancement effects in unknown domains. This dynamic processing method effectively compensates for the limitations of insufficient data during the training phase, while improving the model's adaptability and robustness to new domains.

[0084] Compared to existing methods, this operation effectively optimizes model performance by dynamically utilizing existing training data during inference, combining frequency domain transformation and fine-tuning strategies. This significantly improves the model's generalization ability and adaptability in unknown domains. When the model infers in a new, unknown domain, a frequency domain distribution alignment operation is performed to generate labeled data that more closely matches the distribution of the new domain. Then, the inference data is combined with the distribution-aligned memory data to further optimize the model. This process is similar to fine-tuning, eliminating the need to rely on ground truth calculations of the loss function based on the new domain data; loss calculations can be performed using only the distribution-aligned labeled data. Through several simple optimization iterations, the model can quickly adjust its parameters to better adapt to new, unknown domains, thereby achieving more efficient image enhancement performance.

[0085] It is worth noting that although the above description pertains to supervised learning methods, its applicability is not limited to this. The process described can also be applied to unsupervised and self-supervised learning scenarios. Therefore, this method has broad applicability and can improve model performance across various learning paradigms.

[0086] The multi-domain image enhancement device based on memory and decision fusion provided by the present invention is described below. The multi-domain image enhancement device based on memory and decision fusion described below can be referred to in correspondence with the multi-domain image enhancement method based on memory and decision fusion described above.

[0087] Figure 4 This is a schematic diagram of the structure of the multi-domain image enhancement device based on memory and decision fusion provided by the present invention, as shown below. Figure 4 As shown, this multi-domain image enhancement device based on memory and decision fusion includes: The extraction module 401 is configured to extract at least one image pair from the memory unit when receiving an image to be enhanced in an unknown region, the image pair including the original image of the known region and the enhancement image label corresponding to the original image; Alignment module 402 is configured to call the domain alignment unit in the decision unit to align the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images; The optimization module 403 is configured to optimize the image enhancement unit in the decision unit based on the aligned image and the enhanced image label corresponding to each of the original images; The image enhancement module 404 is configured to perform image enhancement on the image to be enhanced based on the optimized image enhancement unit, so as to obtain the enhanced image corresponding to the image to be enhanced.

[0088] The multi-domain image enhancement device based on memory and decision fusion provided by this invention can receive image input from different domains. By introducing a memory unit to store existing image pairs and combining it with a domain alignment unit in the decision unit, it adaptively aligns the image to be enhanced with each original image in the frequency domain, thereby constructing labeled data for unknown new domains and reducing data labeling costs. Furthermore, the image enhancement unit is optimized based on the aligned image and the labels of the enhanced image, ensuring cross-domain image enhancement performance and ensuring that the enhancement process can adapt to the specific needs of different domains. The fusion of memory and decision not only significantly improves the quality of image enhancement but also optimizes the computational efficiency and cross-domain adaptability of the model, surpassing the performance of existing technologies in complex multi-domain scenarios.

[0089] In one or more optional embodiments of the present invention, the alignment module 402 is specifically configured as follows: For each original image, Fourier transform, frequency domain transform, and inverse Fourier transform are performed on the original image and the image to be enhanced to obtain the aligned image corresponding to the original image.

[0090] In one or more optional embodiments of the present invention, the alignment module 402 is specifically configured as follows: Fourier transform is performed on the original image and the image to be enhanced, respectively, to obtain the frequency domain features of the original image and the frequency domain features of the image to be enhanced; A distribution alignment operation is performed on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features; Perform an inverse Fourier transform on the aligned frequency domain features to obtain the aligned image corresponding to the original image.

[0091] In one or more optional embodiments of the present invention, the frequency domain features include a phase spectrum and an amplitude spectrum; The alignment module 402 is specifically configured as follows: The central region of the amplitude spectrum of the original image is replaced with the central region of the amplitude spectrum of the image to be enhanced, while keeping the non-central region of the amplitude spectrum of the original image and the phase spectrum of the original image unchanged, to obtain the aligned frequency domain features. The size and shape of the central region are set based on task requirements.

[0092] In one or more optional embodiments of the present invention, the extraction module 401 is specifically configured as follows: Obtain the domain distribution parameters and the first quantity of the image to be enhanced; Based on the first quantity, determine the second quantity; Based on the domain distribution parameters of the image to be enhanced and the domain distribution parameters of each image pair stored in the memory unit, the second number of image pairs are extracted from each image pair stored in the memory unit.

[0093] In one or more optional embodiments of the present invention, the multi-domain image enhancement device based on memory and decision fusion further includes a preprocessing module configured to: The image to be enhanced is preprocessed, including normalization and / or filtering.

[0094] In one or more optional embodiments of the present invention, the multi-domain image enhancement device based on memory and decision fusion further includes a storage module configured to: The aligned image corresponding to the original image and the enhanced image label of the original image are determined as a new image pair, and the new image pair is stored in the memory unit; And / or, determine the image to be enhanced and the enhanced image as a new image pair, and store the new image pair in the memory unit.

[0095] In one or more optional embodiments of the present invention, the storage module is configured to: The domain distribution parameters of the original image in the new image pair and the new image pair are stored in the memory unit in a structured form.

[0096] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a multi-domain image enhancement method based on memory and decision fusion. This method includes: upon receiving an image to be enhanced in an unknown domain, extracting at least one image pair from the memory unit, the image pair including an original image in a known domain and an enhanced image label corresponding to the original image; calling a domain alignment unit in a decision unit to align the image to be enhanced with each of the original images to obtain aligned images corresponding to each of the original images; optimizing the image enhancement unit in the decision unit based on the aligned images corresponding to each of the original images and the enhanced image labels; and performing image enhancement on the image to be enhanced based on the optimized image enhancement unit to obtain an enhanced image corresponding to the image to be enhanced.

[0097] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the multi-domain image enhancement method based on memory and decision fusion provided by the above methods. The method includes: upon receiving an image to be enhanced in an unknown domain, extracting at least one image pair from a memory unit, the image pair including an original image in a known domain and an enhanced image label corresponding to the original image; invoking a domain alignment unit in a decision unit to align the image to be enhanced with each of the original images to obtain aligned images corresponding to each of the original images; optimizing the image enhancement unit in the decision unit based on the aligned images corresponding to each of the original images and the enhanced image label; and performing image enhancement on the image to be enhanced based on the optimized image enhancement unit to obtain an enhanced image corresponding to the image to be enhanced.

[0099] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the multi-domain image enhancement method based on memory and decision fusion provided by the above methods. The method includes: upon receiving an image to be enhanced in an unknown domain, extracting at least one image pair from a memory unit, the image pair including an original image in a known domain and an enhanced image label corresponding to the original image; invoking a domain alignment unit in a decision unit to align the image to be enhanced with each of the original images to obtain aligned images corresponding to each of the original images; optimizing an image enhancement unit in the decision unit based on the aligned images corresponding to each of the original images and the enhanced image label; and performing image enhancement on the image to be enhanced based on the optimized image enhancement unit to obtain an enhanced image corresponding to the image to be enhanced.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-domain image enhancement method based on the fusion of memory and decision-making, characterized in that, include: Upon receiving an image to be enhanced in an unknown region, at least one image pair is extracted from the memory unit, the image pair including the original image of the known region and the enhanced image label corresponding to the original image; The domain alignment unit in the decision unit is invoked to align the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images; Based on the aligned image and the enhanced image label corresponding to each of the original images, the image enhancement unit in the decision unit is optimized; Based on the optimized image enhancement unit, the image to be enhanced is enhanced to obtain the enhanced image corresponding to the image to be enhanced.

2. The multi-domain image enhancement method based on memory and decision fusion according to claim 1, characterized in that, The step of aligning the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images includes: For each original image, Fourier transform, frequency domain transform, and inverse Fourier transform are performed on the original image and the image to be enhanced to obtain the aligned image corresponding to the original image.

3. The multi-domain image enhancement method based on memory and decision fusion according to claim 2, characterized in that, The step of performing Fourier transform, frequency domain transform, and inverse Fourier transform on the original image and the image to be enhanced to obtain the aligned image corresponding to the original image includes: Fourier transform is performed on the original image and the image to be enhanced, respectively, to obtain the frequency domain features of the original image and the frequency domain features of the image to be enhanced; A distribution alignment operation is performed on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features; Perform an inverse Fourier transform on the aligned frequency domain features to obtain the aligned image corresponding to the original image.

4. The multi-domain image enhancement method based on memory and decision fusion according to claim 3, characterized in that, The frequency domain features include the phase spectrum and the amplitude spectrum; The step of performing a distribution alignment operation on the frequency domain features of the original image and the frequency domain features of the image to be enhanced to obtain aligned frequency domain features includes: The central region of the amplitude spectrum of the original image is replaced with the central region of the amplitude spectrum of the image to be enhanced, while keeping the non-central region of the amplitude spectrum of the original image and the phase spectrum of the original image unchanged, to obtain the aligned frequency domain features. The size and shape of the central region are set based on task requirements.

5. The multi-domain image enhancement method based on memory and decision fusion according to claim 1, characterized in that, The step of retrieving at least one image pair from the memory unit upon receiving an image to be enhanced in an unknown region includes: Obtain the domain distribution parameters and the first quantity of the image to be enhanced; Based on the first quantity, determine the second quantity; Based on the domain distribution parameters of the image to be enhanced and the domain distribution parameters of each image pair stored in the memory unit, the second number of image pairs are extracted from each image pair stored in the memory unit.

6. The multi-domain image enhancement method based on memory and decision fusion according to any one of claims 1-5, characterized in that, Before aligning the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images, the method further includes: The image to be enhanced is preprocessed, including normalization and / or filtering.

7. The multi-domain image enhancement method based on memory and decision fusion according to any one of claims 1-5, characterized in that, The method further includes: The aligned image corresponding to the original image and the enhanced image label of the original image are determined as a new image pair, and the new image pair is stored in the memory unit; And / or, determine the image to be enhanced and the enhanced image as a new image pair, and store the new image pair in the memory unit.

8. The multi-domain image enhancement method based on memory and decision fusion according to claim 7, characterized in that, The step of storing the new image pair into the memory unit includes: The domain distribution parameters of the original image in the new image pair and the new image pair are stored in the memory unit in a structured form.

9. A multi-domain image enhancement device based on memory and decision fusion, characterized in that, include: The extraction module is configured to extract at least one image pair from a memory unit upon receiving an image to be enhanced in an unknown region, the image pair including the original image of the known region and the enhancement image label corresponding to the original image; The alignment module is configured to call the domain alignment unit in the decision unit to align the image to be enhanced with each of the original images to obtain the aligned image corresponding to each of the original images; The optimization module is configured to optimize the image enhancement unit in the decision unit based on the aligned image and the enhanced image label corresponding to each of the original images; The image enhancement module is configured to perform image enhancement on the image to be enhanced based on the optimized image enhancement unit, so as to obtain the enhanced image corresponding to the image to be enhanced.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the multi-domain image enhancement method based on memory and decision fusion as described in any one of claims 1 to 8.

Images