Image processing method and device, electronic equipment and storage medium
By acquiring high-resolution and low-resolution images, generating mask maps and processing images using filter mapping coefficients, the problem of not being able to completely preserve the original image integrity in the prior art is solved, and efficient molar removal and image quality improvement are achieved.
Patent Information
- Application Number
- CN202510192425.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-20
- Publication Date
- 2025-05-16
AI Technical Summary
Existing molar removal methods cannot completely preserve the integrity of the original image while removing molar patterns.
By acquiring a high-resolution first image and a low-resolution second image, a mask map of the molar region in the image is generated, and the first image is processed according to the filter mapping coefficient to obtain a target image, thereby realizing the removal of molar patterns.
It realizes the important information of the image while removing molar patterns, improves the quality of the image, and supports real-time molar removal of high-resolution images.
Smart Images
Figure CN120013768A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of image processing and image moiré removal technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art
[0002] Moiré is an interference pattern that often appears in digital images. When the spatial frequency of the pixels of the photosensitive element is close to the spatial frequency of the stripes in the image, a new wavy interference pattern may be generated, usually appearing as ripples, stripes or other periodic patterns. Existing methods for removing moiré have some limitations in practical applications and cannot completely preserve the integrity of the original image while removing moiré. Summary of the invention
[0003] The present disclosure provides an image processing method, an apparatus, an electronic device, and a storage medium.
[0004] According to one aspect of the present disclosure, there is provided an image processing method, including: acquiring a first image, and acquiring a second image based on the first image, wherein the resolution of the first image is higher than the resolution of the second image; acquiring a mask map of a moiré area in the image based on the first image or the second image; acquiring filter mapping coefficients based on the first image and the second image, and acquiring a third image based on the filter mapping coefficients and the first image; processing the first image based on the third image and the mask map to obtain a target image.
[0005] According to another aspect of the present disclosure, there is provided an image processing device, including: a first acquisition module, used to acquire a first image, and acquire a second image based on the first image, wherein the resolution of the first image is higher than the resolution of the second image; a second acquisition module, used to acquire a mask map of a moiré area in an image based on the first image or the second image; a third acquisition module, used to acquire a filter mapping coefficient based on the first image and the second image, and acquire a third image based on the filter mapping coefficient and the first image; and a processing module, used to process the first image based on the third image and the mask map to obtain a target image.
[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method described in the above-mentioned one aspect embodiment.
[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, on which a computer program / instructions are stored, and the computer instructions are used to enable the computer to execute the image processing method described in the above-mentioned embodiment.
[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the image processing method described in the above-mentioned embodiment is implemented.
[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0011] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;
[0012] Figure 2 A flowchart of another image processing method provided by an embodiment of the present disclosure;
[0013] Figure 3 A flowchart of another image processing method provided by an embodiment of the present disclosure;
[0014] Figure 4 A flowchart of another image processing method provided by an embodiment of the present disclosure;
[0015] Figure 5 A schematic diagram of a process for removing moiré from an image provided by an embodiment of the present disclosure;
[0016] Figure 6 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;
[0017] Figure 7 The present invention is a block diagram of an electronic device for implementing the image processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0019] The image processing method, apparatus, electronic device, and storage medium according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.
[0020] Artificial Intelligence (AI) is a discipline that studies how computers can simulate certain thought processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technologies and software-level technologies. AI hardware technologies generally include computer vision technology, speech recognition technology, natural language processing technology, as well as learning / deep learning, big data processing technology, knowledge graph technology, and other aspects.
[0021] Image processing refers to the technology of using computers to analyze images to achieve the desired results. Image processing generally refers to digital image processing. Digital images refer to a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of the array are called pixels, and their values are called grayscale values. Image processing technology generally includes three parts: image compression, enhancement and restoration, matching, description and recognition.
[0022] Figure 1 A flowchart of an image processing method provided in an embodiment of the present disclosure is provided.
[0023] like Figure 1 As shown, the image processing method may include:
[0024] S101, acquiring a first image, and acquiring a second image according to the first image, wherein a resolution of the first image is higher than a resolution of the second image.
[0025] It should be noted that the execution subject of the image processing method in the embodiment of the present disclosure may be a hardware device with data processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a server, a user terminal and other intelligent devices. Optionally, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, etc. Optionally, the server includes but is not limited to a network server, an application server, and may also be a server of a distributed system, or a server combined with a blockchain, etc. The embodiment of the present disclosure is not specifically limited.
[0026] In some embodiments, an image containing moiré patterns may be used as the first image.
[0027] In some embodiments, an image with moiré patterns can be selected from an image library as the first image, and / or an image with moiré patterns can be downloaded from the Internet as the first image, and / or an image captured by an image capture device can be used as the first image.
[0028] In some embodiments, the first image may be downsampled to obtain the second image, so that the resolution of the first image is higher than the resolution of the second image.
[0029] S102: Acquire a mask image of a moiré region in the image according to the first image or the second image.
[0030] In some embodiments, the moiré pattern may be identified and segmented on the first image or the second image to obtain a segmented image of the moiré pattern area, and the segmented image may be further binarized to obtain a mask image of the moiré pattern area.
[0031] In some embodiments, a moiré region segmentation model can be pre-trained. By inputting the first image or the second image into the moiré region segmentation model, the moiré region segmentation model can identify and segment the moiré and binarize the segmented image to output a mask map of the moiré region.
[0032] In some embodiments, the moiré region segmentation model may be a segmentation network model of any structure, and the embodiments of the present disclosure do not specifically limit the moiré region segmentation model.
[0033] S103, obtaining filter mapping coefficients according to the first image and the second image, and obtaining a third image according to the filter mapping coefficients and the first image.
[0034] In some embodiments, a high-resolution first high-frequency guide map can be determined based on the first image, and a low-resolution second high-frequency guide map can be determined based on the first high-frequency guide map. Furthermore, a mean filtering operation can be performed on the second image and the second high-frequency guide map to obtain a corresponding filtered image, so that the filter mapping coefficient can be determined based on the covariance information and variance information of the filtered image.
[0035] In some embodiments, after the filter mapping coefficients are obtained, the first image is enhanced and noise is added based on the filter mapping coefficients to obtain a third image.
[0036] In some embodiments, mean filtering operations may be performed on the second image, the second high-frequency guide map, the product of the second image and the second high-frequency guide map, and the product of the second image and the second image, respectively, to obtain corresponding filtered images.
[0037] S104: Process the first image according to the third image and the mask image to obtain a target image.
[0038] In some embodiments, the first image and the third image may be fused based on the mask image to process the first image and obtain a target image with moiré removed.
[0039] In some embodiments, a first pixel belonging to a moiré region in a third image can be determined based on a mask image, and a second pixel belonging to a moiré region in a first image can be determined, and then the second pixel can be replaced with the first pixel to process the first image, thereby obtaining a target image.
[0040] In some embodiments, the replacement of pixels may be performed based on the pixel values of the pixels.
[0041] In some embodiments, based on the mask map, pixels marked as moiré regions by the mask map can be identified and extracted from the third image as first pixels. For example, pixels belonging to the moiré region in the mask map are marked as 1, and pixels not belonging to the moiré region are marked as 0, and pixels marked as 1 in the third image are identified as first pixels based on the mask map.
[0042] Furthermore, a pixel corresponding to the first pixel position is determined from the first image as the second pixel, and the pixel value of the second pixel is replaced with the pixel value of the first pixel, thereby removing the moiré pattern in the first image and obtaining the target image.
[0043] According to the image processing method provided by the embodiment of the present disclosure, a mask map of the moiré region in the image is determined by acquiring a first image and determining a second image having a lower resolution than the first image based on the first image. Further, a filter mapping coefficient is determined based on the first image and the second image, so as to map the first image based on the filter mapping coefficient to obtain a third image, and then the first image can be processed based on the third image and the mask map to obtain a target image with the moiré removed, thereby achieving real-time moiré removal of high-resolution images, ensuring the integrity of the image, and improving the image quality. The present disclosure can be applied to different software, thereby improving the work effect of photo editing.
[0044] Figure 2 A flowchart of an image processing method provided in an embodiment of the present disclosure is provided.
[0045] like Figure 2 As shown, the image processing method may include:
[0046] S201, acquiring a first image, and acquiring a second image according to the first image, wherein a resolution of the first image is higher than a resolution of the second image.
[0047] The relevant contents of step S201 can be found in the above embodiment and will not be described again here.
[0048] S202: Acquire a mask image of a moiré region in the image according to the first image or the second image.
[0049] In some embodiments, by performing moiré recognition and segmentation on any one of the first image and the second image, determining a segmented image of the moiré area, and binarizing the segmented image of the moiré area, a mask map of the moiré area is obtained, thereby accurately extracting the moiré area in the image, further simplifying the moiré removal process and improving the efficiency of image processing.
[0050] In some embodiments, a pre-trained moiré region segmentation model is obtained, and either the first image or the second image is input into the moiré region segmentation model. The moiré region segmentation model performs moiré recognition and segmentation on either image to obtain a segmented image of the moiré region, and the segmented image is binarized to output a mask image of the moiré region.
[0051] S203: Acquire a first high-frequency guide map and a second high-frequency guide map according to the first image, wherein a resolution of the first high-frequency guide map is higher than a resolution of the second high-frequency guide map.
[0052] In some embodiments, a first high-resolution high-frequency guide map can be determined according to the first image, and a second high-frequency guide map with low resolution can be determined based on the first high-frequency guide map, so that the resolution of the first high-frequency guide map is higher than that of the second high-frequency guide map.
[0053] In some embodiments, by performing a convolution operation on the first image to obtain a first high-frequency guide map, key features in the image can be effectively extracted and image details can be enhanced. The first high-frequency guide map is then downsampled to obtain a second high-frequency guide map, thereby reducing the size and complexity of the image and further improving the efficiency of subsequent image processing.
[0054] In some embodiments, a convolution operation is performed on the first image to obtain a first high-frequency guide map using the following formula:
[0055]
[0056] Among them, K is the convolution kernel, I hg (x, y) is the pixel value of the first high-frequency guide image at position (x, y), I hr (x+i, y+j) is the pixel value of the first image at position (x+i, y+j), K(i, j) is the value of the convolution kernel at position (i, j), and m and n are the radius of the convolution kernel.
[0057] S204: Obtain filter mapping coefficients according to the second image and the second high-frequency guide map.
[0058] In some embodiments, filtering is performed on the second image and the second high-frequency guide map to obtain respective filtered images, and then the filtering mapping coefficients can be determined based on the filtered images. The second image and the second high-frequency guide map are low-resolution images. Since the filtering process is performed on the low-resolution images, the time consumption and memory usage can be effectively controlled.
[0059] In some embodiments, the second image and the second high-frequency guide map may be mean filtered to obtain respective filtered images.
[0060] In some embodiments, a mean filtering operation is performed on the second image to obtain a first filtered image; and a mean filtering operation is performed on the second high-frequency guide image to obtain a second filtered image.
[0061] In some embodiments, the third filtered image is obtained by multiplying the second image and the second high-frequency guide map pixel by pixel and then performing a mean filtering operation, and the fourth filtered image is obtained by multiplying the second image pixel by pixel and then performing a mean filtering operation.
[0062] Optionally, the formula for calculating the first filtered image to the fourth filtered image is as follows:
[0063]
[0064] Among them, mean_I lg (x,y),mean_I lr (x,y), mean_lg lg (x,y),mean_II lg (x, y) represents the pixel at position (x, y) of the first filtered image, the second filtered image, the third filtered image, and the fourth filtered image, respectively. Represents the size of the convolution kernel element, I lr (x+i,y+j),l lg (x+i, y+j) represent the pixel values of the second image and the second high-frequency guide map at the position (x+i, y+j), respectively.
[0065] Furthermore, filter mapping coefficients may be acquired according to the first filter image to the fourth filter image.
[0066] In some embodiments, covariance information and variance information between the image and the high-frequency guidance image can be obtained based on the first to fourth filtered images, so that the filter mapping coefficients can be determined based on the covariance information and variance information, which can enhance the adaptability of the filter mapping and improve the accuracy of image processing.
[0067] In some embodiments, covariance information may be calculated based on the first filtered image, the second filtered image, and the third filtered image, so that the covariance information can reflect the correlation between the filtered images and optimize the effect of image mapping using the filter mapping coefficients.
[0068] In some embodiments, the first filtered image and the second filtered image are multiplied pixel by pixel to obtain a first multiplied image, and the third filtered image and the first multiplied image are subtracted to obtain covariance information.
[0069] Optionally, the formula for calculating covariance information is as follows:
[0070] cov_Ig=mean_lg lg -mean_I lg *mean_I lr (3)
[0071] Among them, cov_Ig represents covariance information, mean_I lg *mean_I lr Represents the first multiplied image, mean_lg lg represents the third filtered image.
[0072] In some embodiments, variance information may be calculated based on the first filtered image and the fourth filtered image, so that the variance information can optimize the filter mapping coefficients and improve the effect of image mapping using the filter mapping coefficients.
[0073] In some embodiments, the second multiplied image is obtained by performing pixel multiplication on the first filtered image, and the variance information is obtained by performing a subtraction operation on the fourth filtered image and the second multiplied image.
[0074] Optionally, the formula for calculating variance information is as follows:
[0075] var_I=mean_II lg -mean_I lg *mean_I lg (4)
[0076] Among them, var_I represents variance information, mean_I lg *mean_I lg Represents the second multiplied image, mean_II lg represents the fourth filtered image.
[0077] In some embodiments, the filter mapping coefficients include high-resolution similarity coefficients and offset coefficients, and a first similarity coefficient can be determined based on covariance information and variance information, and a first offset coefficient can be determined based on the first filtered image, the third filtered image and the first similarity coefficient.
[0078] Furthermore, in order to improve the resolution of the first similarity coefficient and the first offset coefficient, upsampling is performed on the first similarity coefficient and the first offset coefficient to obtain filter mapping coefficients, that is, the first similarity coefficient and the first offset coefficient are upsampled to obtain high-resolution second similarity coefficient and second offset coefficient as filter mapping coefficients.
[0079] In some embodiments, the resolution of the second similarity coefficient and the second offset coefficient is consistent with the resolution of the first image, so that the images can be accurately matched and positioned, thereby more accurately positioning the moiré region and improving the moiré removal effect.
[0080] In some embodiments, according to the covariance information and the variance information, the formula for calculating the first similarity coefficient is as follows:
[0081] a k =cov_Ig / (var_I+esp)(5)
[0082] Among them, a k represents the first similarity coefficient, cov_Ig represents covariance information, var_I represents variance information, and esp represents parameters.
[0083] In some embodiments, according to the first filtered image, the third filtered image and the first similarity coefficient, the formula for calculating the first offset coefficient is as follows:
[0084] b k =mean_I lg -a k *mean_lg lg (6)
[0085] Among them, b k Represents the first offset coefficient, mean_I lg Represents the first filtered image, mean_lg lg represents the third filtered image, a k Represents the first similarity coefficient.
[0086] S205: Acquire a third image according to the filter mapping coefficients and the first image.
[0087] S206: Process the first image according to the third image and the mask image to obtain a target image.
[0088] The relevant contents of steps S205-S206 can be found in the above embodiment and will not be repeated here.
[0089] According to the image processing method provided by the embodiment of the present disclosure, by acquiring the first high-frequency guide map and the second high-frequency guide map, and determining the filter mapping coefficient according to the second image and the second high-frequency guide map, it is possible to achieve accurate control of the image filter mapping. According to the high-frequency guide map, the characteristics of the moiré pattern can be accurately captured, providing accurate information for subsequent moiré pattern removal.
[0090] Figure 3 A flowchart of an image processing method provided in an embodiment of the present disclosure is provided.
[0091] like Figure 3 As shown, the image processing method may include:
[0092] S301, down-sample the first image to obtain a fifth image, and perform floating point number format and normalization processing on the fifth image to obtain a second image.
[0093] In some embodiments, in order to obtain a low-resolution second image, the first image can be downsampled to obtain a low-resolution fifth image, and then the fifth image is processed in floating-point format and normalized to obtain a second image with higher precision and low resolution to ensure the accuracy of the calculation.
[0094] For example, the fifth image may be processed into a 32-bit floating point format and normalized to obtain the second image.
[0095] S302: Acquire a mask image of a moiré region in the image according to the first image or the second image.
[0096] S303: Obtain filter mapping coefficients according to the first image and the second image.
[0097] The relevant contents of steps S302 - S303 can be found in the above embodiment and will not be described again here.
[0098] S304: Perform edge enhancement processing on the first image according to the filter mapping coefficients to obtain a fourth image.
[0099] In some embodiments, the filter mapping coefficients can be used to map the first image based on the first high-frequency guidance map to achieve edge enhancement of the first image, thereby obtaining a fourth image. The filter mapping coefficients use high-frequency information for guidance, which can achieve precise edge enhancement and reduce image distortion.
[0100] In some embodiments, the filter mapping coefficients include a second similarity coefficient and a second offset coefficient, and the resolution of the second similarity coefficient and the second offset coefficient is higher than the resolution of the first similarity coefficient and the first offset coefficient.
[0101] In some embodiments, the first high-frequency guide map of the first image can be multiplied based on the second similarity coefficient to obtain a third multiplied image, and the third multiplied image can be offset based on the second offset coefficient to obtain a fourth image, thereby realizing the filtering mapping coefficient and performing edge enhancement processing on the first image.
[0102] Optionally, the formula for edge enhancement processing of the first image using the filter mapping coefficient is as follows:
[0103] I ho =A k *I hg +B k (7)
[0104] Among them, I ho Represents the fourth image, A k Represents the second similarity coefficient, A k *I hg represents the third multiplied image, B k Represents the second offset coefficient.
[0105] S305: Add noise conforming to the normal distribution to the fourth image to obtain a third image.
[0106] In some embodiments, adding noise to an image can prevent the generation of new moiré patterns caused by compressing the image. An array of the same size as the first image can be randomly generated, and the array contains normally distributed random numbers, which are used as noise that conforms to the normal distribution, and then the noise is added to the fourth image to obtain the third image.
[0107] Optionally, the formula for adding noise is as follows:
[0108] I hn =N h +I ho (8)
[0109] Among them, I hn represents the third image, N h represents the noise that conforms to the normal distribution, I ho Indicates the fourth image.
[0110] For example, each element of a normally distributed random number with a mean of 0 and a standard deviation of 0.007 can be used as the noise that conforms to the normal distribution.
[0111] In some embodiments, the third image can also be processed in a floating-point data format to reduce the floating-point bits of the third image. This can reduce the running time and computing resources of the moiré removal process while ensuring the moiré removal effect, and further reduce the memory and video memory usage.
[0112] For example, the third image is processed into an 8-bit floating point number image.
[0113] S306: Process the first image according to the third image and the mask image to obtain a target image.
[0114] The relevant contents of step S306 can be found in the above embodiment and will not be described again here.
[0115] According to the image processing method provided by the embodiment of the present disclosure, the first image is edge enhanced according to the filter mapping coefficient to obtain a fourth image, and the third image is obtained by adding noise to the fourth image. This can prevent the compression of the image from causing new moiré patterns to be generated, thereby ensuring the effect of removing moiré patterns.
[0116] Figure 4 A flowchart of an image processing method provided in an embodiment of the present disclosure is provided.
[0117] like Figure 4 As shown, the image processing method may include:
[0118] S401, acquiring a first image, and acquiring a second image according to the first image, wherein a resolution of the first image is higher than a resolution of the second image.
[0119] S402: Acquire a mask image of a moiré region in the image according to the first image or the second image.
[0120] S403, acquiring filter mapping coefficients according to the first image and the second image, and acquiring a third image according to the filter mapping coefficients and the first image.
[0121] The relevant contents of steps S401 - S403 can be found in the above embodiment and will not be described again here.
[0122] S404: extracting a first pixel belonging to a moiré region from the third image based on the mask image.
[0123] S405: Replace the second pixel in the first image that belongs to the moiré region based on the position and pixel value of the first pixel to obtain a target image.
[0124] In some embodiments, by determining a first pixel in the third image that belongs to the moiré region and determining a second pixel in the first image that belongs to the moiré region, the second pixel may be replaced with the first pixel to obtain a target image.
[0125] In some embodiments, the first pixel belonging to the moiré region in the third image can be extracted based on the mask image. For example, the moiré region in the mask image is marked as 1, and the region not belonging to the moiré region is marked as 0. The mask image is overlaid on the third image, and the region marked as 1 in the third image can be determined, and the pixel marked as 1 in the region of the third image is extracted as the first pixel.
[0126] Furthermore, the position and pixel value of the first pixel are determined, and a pixel corresponding to the position of the first pixel is determined from the first image as the second pixel, and then the pixel value of the second pixel is replaced using the pixel value of the first pixel to obtain a target image with moire removed.
[0127] For example, if the first pixel is at position A, position B, and position C, the pixel values of the first pixel corresponding to different positions are pixel value 1, pixel value 12, and pixel value 3, respectively. Position A corresponds to pixel A in the first image, position B corresponds to pixel B in the first image, and position C corresponds to pixel C in the first image. Then, the pixel value of pixel A can be replaced with pixel value 1, the pixel value of pixel B can be replaced with pixel value 2, and the pixel value of pixel C can be replaced with pixel value 3, so as to obtain the first image with replaced pixel values as the target image for removing moire.
[0128] According to the image processing method provided by the embodiment of the present disclosure, a first pixel belonging to a moiré area is extracted from a third image based on a mask image, and a second pixel belonging to the moiré area in the first image is replaced based on the position and pixel value of the first pixel to obtain a target image. The moiré can be accurately located, thereby performing targeted replacement. The moiré can be effectively removed while retaining important information of the image, thereby further improving the image quality of the target image.
[0129] Figure 5 Shown is a flow chart for removing moiré from an image.
[0130] By obtaining a high-resolution image with moiré as the first image, and determining a second image obtained by downsampling the first image, that is, the resolution of the first image is higher than the resolution of the second image. By inputting any one of the first image and the second image into the moiré segmentation model, a mask map of the moiré area can be output. By performing a convolution operation on the first image to obtain a high-resolution first high-frequency guide map, and downsampling the first high-frequency guide map, a second high-frequency guide map can be obtained.
[0131] Further, the filter mapping coefficients can be determined according to the second image and the second high-frequency guide map. The relevant contents of determining the filter mapping coefficients can be referred to the above embodiments, which will not be repeated here. Further, the first image is edge enhanced according to the filter mapping coefficients to obtain a fourth image, and the third image is obtained by adding noise to the fourth image. Further, the first image is processed according to the third image and the mask map to obtain a target image with moiré removed.
[0132] Corresponding to the image processing methods provided in the above-mentioned embodiments, an embodiment of the present disclosure further provides an image processing device. Since the image processing device provided in the embodiment of the present disclosure corresponds to the image processing methods provided in the above-mentioned embodiments, the implementation methods of the above-mentioned image processing methods are also applicable to the image processing device provided in the embodiment of the present disclosure and will not be described in detail in the following embodiments.
[0133] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present disclosure.
[0134] like Figure 6 As shown, the image processing device 600 of the embodiment of the present disclosure includes a first acquisition module 601 , a second acquisition module 602 , a third acquisition module 603 and a processing module 604 .
[0135] A first acquisition module 601 is used to acquire a first image, and acquire a second image according to the first image, wherein the resolution of the first image is higher than the resolution of the second image;
[0136] A second acquisition module 602 is used to acquire a mask image of a moiré region in the image according to the first image or the second image;
[0137] A third acquisition module 603 is used to acquire filter mapping coefficients according to the first image and the second image, and acquire a third image according to the filter mapping coefficients and the first image;
[0138] The processing module 604 is used to process the first image according to the third image and the mask image to obtain a target image.
[0139] In one embodiment of the present disclosure, the third acquisition module 603 is also used to: acquire a first high-frequency guide map and a second high-frequency guide map based on the first image, wherein the resolution of the first high-frequency guide map is higher than the resolution of the second high-frequency guide map; and acquire filter mapping coefficients based on the second image and the second high-frequency guide map.
[0140] In one embodiment of the present disclosure, the third acquisition module 603 is further used to: perform a convolution operation on the first image to obtain a first high-frequency guide map; and downsample the first high-frequency guide map to obtain a second high-frequency guide map.
[0141] In one embodiment of the present disclosure, the third acquisition module 603 is also used to: perform a mean filtering operation on the second image to obtain a first filtered image; perform a mean filtering operation on the second high-frequency guide map to obtain a second filtered image; perform pixel-by-pixel multiplication on the second image and the second high-frequency guide map and then perform a mean filtering operation to obtain a third filtered image; perform pixel-by-pixel multiplication on the second image and then perform a mean filtering operation to obtain a fourth filtered image; and obtain filter mapping coefficients based on the first to fourth filtered images.
[0142] In one embodiment of the present disclosure, the third acquisition module 603 is also used to: acquire covariance information and variance information between the image and the high-frequency guidance image based on the first filtered image to the fourth filtered image; determine a first similarity coefficient based on the covariance information and the variance information; determine a first offset coefficient based on the first filtered image, the third filtered image and the first similarity coefficient; and upsample according to the first similarity coefficient and the first offset coefficient to obtain a filter mapping coefficient.
[0143] In one embodiment of the present disclosure, the third acquisition module 603 is further used to: multiply the first filtered image and the second filtered image pixel by pixel to obtain a first multiplied image; and subtract the third filtered image from the first multiplied image to obtain covariance information.
[0144] In one embodiment of the present disclosure, the third acquisition module 603 is further used to: perform pixel self-multiplication on the first filtered image to obtain a second multiplied image; and perform a subtraction operation on the fourth filtered image and the second multiplied image to obtain variance information.
[0145] In one embodiment of the present disclosure, the third acquisition module 603 is further used to: perform edge enhancement processing on the first image according to the filter mapping coefficient to obtain a fourth image; and add noise that conforms to the normal distribution to the fourth image to obtain the third image.
[0146] In one embodiment of the present disclosure, the third acquisition module 603 is also used to: filter mapping coefficients include a second similarity coefficient and a second offset coefficient, and based on the second similarity coefficient, multiply the first high-frequency guide map of the first image to obtain a third multiplied image; and offset the third multiplied image based on the second offset coefficient to obtain a fourth image.
[0147] In one embodiment of the present disclosure, the second acquisition module 602 is also used to: perform moiré recognition and segmentation on any one of the first image and the second image, determine the segmented image of the moiré area, and binarize the segmented image of the moiré area to obtain a mask image of the moiré area.
[0148] In one embodiment of the present disclosure, the first acquisition module 601 is further used to: downsample the first image to acquire a fifth image, and perform floating point number format and normalization processing on the fifth image to obtain a second image.
[0149] In one embodiment of the present disclosure, the processing module 604 is also used to: extract a first pixel belonging to a moiré area from a third image based on a mask image; and replace a second pixel belonging to the moiré area in the first image based on a position and a pixel value of the first pixel to obtain a target image.
[0150] According to the image processing device provided by the embodiment of the present disclosure, a mask map of the moiré region in the image is determined by acquiring a first image and determining a second image having a lower resolution than the first image based on the first image. Further, a filter mapping coefficient is determined based on the first image and the second image, so as to map the first image based on the filter mapping coefficient to obtain a third image, and then the first image can be processed based on the third image and the mask map to obtain a target image with the moiré removed, thereby achieving real-time moiré removal of high-resolution images, ensuring the integrity of the image, and improving the image quality. The present disclosure can be applied to different software, thereby improving the work effect of photo editing.
[0151] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0152] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0153] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0154] like Figure 7As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program / instruction stored in a read-only memory (ROM) 702 or a computer program / instruction loaded from a storage unit 706 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0155] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, etc.; a storage unit 708 such as a disk, an optical disk, etc.; and a communication unit 709 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0156] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 706. In some embodiments, part or all of the computer program / instructions may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program / instructions are loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).
[0157] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0158] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0159] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0161] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.
[0162] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs / instructions running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the disclosure can be achieved, and this document does not limit them here.
[0164] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. An image processing method, characterized in that: The method comprises: Acquire a first image, and acquire a second image based on the first image, wherein a resolution of the first image is higher than a resolution of the second image; Acquire a mask image of a moiré region in the image according to the first image or the second image; Acquire filter mapping coefficients according to the first image and the second image, and acquire a third image according to the filter mapping coefficients and the first image; The first image is processed according to the third image and the mask image to obtain a target image.
2. The method according to claim 1, wherein: The step of acquiring filter mapping coefficients according to the first image and the second image includes: Acquire a first high-frequency guide map and a second high-frequency guide map according to the first image, wherein a resolution of the first high-frequency guide map is higher than a resolution of the second high-frequency guide map; The filter mapping coefficients are obtained according to the second image and the second high frequency guide map.
3. The method according to claim 2, wherein: The acquiring, according to the first image, a first high-frequency guide map and a second high-frequency guide map comprises: Performing a convolution operation on the first image to obtain the first high-frequency guide map; The first high-frequency guide map is downsampled to obtain the second high-frequency guide map.
4. The method according to claim 2, wherein: The step of acquiring the filter mapping coefficient according to the second image and the second high-frequency guide map comprises: Performing a mean filtering operation on the second image to obtain a first filtered image; Performing a mean filtering operation on the second high-frequency guide image to obtain a second filtered image; Multiplying the second image and the second high-frequency guide map pixel by pixel and then performing a mean filtering operation to obtain a third filtered image; Performing pixel multiplication and mean filtering on the second image to obtain a fourth filtered image; The filter mapping coefficients are obtained according to the first filter image to the fourth filter image.
5. The method according to claim 4, wherein: The obtaining the filter mapping coefficient according to the first filter image to the fourth filter image includes: Acquire covariance information and variance information between the image and the high-frequency guidance image according to the first to fourth filtered images; Determining a first similarity coefficient according to the covariance information and the variance information; determining a first offset coefficient according to the first filtered image, the third filtered image and the first similarity coefficient; Upsampling is performed according to the first similarity coefficient and the first offset coefficient to obtain the filter mapping coefficient.
6. The method according to claim 5, wherein: The process of obtaining the covariance information includes: Multiplying the first filtered image and the second filtered image pixel by pixel to obtain a first multiplied image; A subtraction operation is performed on the third filtered image and the first multiplied image to obtain the covariance information.
7. The method according to claim 5, wherein: The process of obtaining the variance information includes: Performing pixel multiplication on the first filtered image to obtain a second multiplied image; A subtraction operation is performed on the fourth filtered image and the second multiplied image to obtain the variance information.
8. The method according to any one of claims 1 to 7, wherein: The step of acquiring a third image according to the filter mapping coefficients and the first image comprises: performing edge enhancement processing on the first image according to the filter mapping coefficients to obtain a fourth image; Noise conforming to normal distribution is added to the fourth image to obtain the third image.
9. The method according to claim 8, wherein: The step of performing edge enhancement processing on the first image according to the filter mapping coefficient to obtain a fourth image includes: The filter mapping coefficients include a second similarity coefficient and a second offset coefficient, and based on the second similarity coefficient, a first high frequency guide map of the first image is multiplied to obtain a third multiplied image; The third multiplied image is shifted based on the second shift coefficient to obtain the fourth image.
10. The method according to any one of claims 1 to 7, wherein: The process of obtaining the mask image of the moiré region includes: Perform moiré recognition and segmentation on any one of the first image and the second image, determine a segmented image of the moiré area, and perform binarization processing on the segmented image of the moiré area to obtain a mask image of the moiré area.
11. The method according to any one of claims 1 to 7, wherein: The acquiring the second image according to the first image comprises: The first image is downsampled to obtain a fifth image, and the fifth image is processed in a floating point format and normalized to obtain the second image.
12. The method according to any one of claims 1 to 7, wherein: The step of processing the first image according to the third image and the mask image to obtain a target image includes: Based on the mask image, extracting a first pixel belonging to the moiré region from the third image; Based on the position and pixel value of the first pixel, a second pixel belonging to the moiré area in the first image is replaced to obtain the target image.
13. An image processing device, characterized in that: The device comprises: A first acquisition module, used to acquire a first image, and acquire a second image according to the first image, wherein the resolution of the first image is higher than the resolution of the second image; A second acquisition module, used for acquiring a mask image of a moiré region in an image according to the first image or the second image; A third acquisition module, configured to acquire filter mapping coefficients according to the first image and the second image, and acquire a third image according to the filter mapping coefficients and the first image; A processing module is used to process the first image according to the third image and the mask image to obtain a target image.
14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.
15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-12.
16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.