Method, device and equipment for eliminating moire patterns and computer readable storage medium
By deploying the left pixel point and the right pixel point on the image sensor, and using the defocus value to determine and eliminate molar marks, the loss of image details caused by molar mark removal in the prior art is solved, and accurate molar mark removal is achieved.
Patent Information
- Application Number
- CN202410031050.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-08
- Publication Date
- 2025-07-08
AI Technical Summary
The prior art tends to lose image details when removing molar marks, and the inability to accurately locate and eliminate molar marks.
The left pixel point and the right pixel point are deployed on the image sensor. By acquiring the defocus values of the left image data and the right image data, it is determined whether there is a molar mark in the image area, and only eliminates it in the area where the molar mark exists.
Accurately position and eliminate molar patterns to avoid details loss caused by processing the entire image, and achieve efficient elimination of molar patterns while retaining image details.
Smart Images
Figure CN120281858A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of communication technologies, and particularly to a method, apparatus, device, and computer-readable storage medium for eliminating moiré patterns. Background Art
[0002] When the spatial frequency of the pixels of an image sensor is close to the spatial frequency of the stripes in the acquired image, a wavy interference pattern may be generated in the image. This wavy interference pattern is called a moiré pattern.
[0003] In related technologies, moiré patterns can be removed by adding a low-pass filter lens or using image filtering techniques on the acquired image. However, filtering the entire moiré-patterned image without screening will result in loss of image details in the picture for these moiré-pattern removal methods. Summary of the Invention
[0004] To overcome the problems in the related technologies, the present disclosure provides a method, apparatus, device, and computer-readable storage medium for eliminating moiré patterns, which can solve the above problems.
[0005] According to a first aspect of an embodiment of the present disclosure, a method for eliminating moiré patterns is provided. Left pixel points and right pixel points are deployed on an image sensor. The left pixel points are used to collect left-side light to generate left-image data when the image sensor acquires a target image, and the right pixel points are used to collect right-side light to generate right-image data when the image sensor acquires a target image. The method includes:
[0006] Obtaining a left-image data image block and a right-image data image block corresponding to at least a part of the region of the target image;
[0007] Determining whether there is a moiré pattern on the at least a part of the region according to the defocus values of the left-image data image block and the right-image data image block;
[0008] When there is a moiré pattern on the at least a part of the region, eliminating the moiré pattern on the target image for the at least a part of the region.
[0009] According to a second aspect of an embodiment of the present disclosure, an apparatus for eliminating moiré patterns is provided. Left pixel points and right pixel points are deployed on an image sensor. The left pixel points are used to collect left-side light to generate left-image data when the image sensor acquires a target image, and the right pixel points are used to collect right-side light to generate right-image data when the image sensor acquires a target image. The apparatus includes:
[0010] An obtaining unit, configured to obtain a left-image data image block and a right-image data image block corresponding to at least a part of the region of the target image;
[0011] A determination unit configured to determine whether moiré exists on at least a part of the region according to the defocus values of the left image data block and the right image data block;
[0012] An elimination unit configured to eliminate moiré on the target image for at least a part of the region when moiré exists on at least a part of the region.
[0013] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device including: a processor and a memory;
[0014] The memory is used to store a computer program;
[0015] The processor is configured to execute the method for eliminating moiré as described in the first aspect by calling the computer program.
[0016] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for eliminating moiré as described in the first aspect is implemented.
[0017] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:
[0018] In the present disclosure, left pixel points and right pixel points are deployed on an image sensor. By PD (Phase Detector) point detection, the defocus value is determined, and then it is determined whether moiré exists on at least a part of the region according to the determined defocus value. When moiré exists, only the region where moiré exists is specifically eliminated. Through the method of the present disclosure, the determined position of moiré in the target image can be located, and only the position where moiré is located is eliminated, thereby avoiding the loss of picture details caused by eliminating the entire target image, and being able to more accurately eliminate moiré while ensuring the image detail information.
[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the present disclosure, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.
[0021] Figure 1A FIG. is a schematic diagram of an image including moiré according to an exemplary embodiment of the present disclosure.
[0022] Figure 1BIt is a schematic diagram of an image without moiré patterns shown according to an exemplary embodiment of the present disclosure.
[0023] Figure 2 It is a schematic flowchart of a method for eliminating moiré patterns shown according to an exemplary embodiment of the present disclosure.
[0024] Figure 3 It is a schematic diagram of the change data of similarity with the interval distance shown according to an exemplary embodiment of the present disclosure.
[0025] Figure 4 It is a schematic diagram of the change data of similarity with the interval distance shown according to an exemplary embodiment of the present disclosure.
[0026] Figure 5 It is a schematic flowchart of a method for determining whether moiré patterns exist on at least a part of the region shown according to an exemplary embodiment of the present disclosure.
[0027] Figure 6 It is a schematic flowchart of a method for eliminating moiré patterns shown according to an exemplary embodiment of the present disclosure.
[0028] Figure 7 It is a schematic diagram of an image processing process shown according to an exemplary embodiment of the present disclosure.
[0029] Figure 8 It is a block diagram of a device for eliminating moiré patterns shown according to an exemplary embodiment of the present disclosure.
[0030] Figure 9 It is a schematic block diagram of a device for eliminating moiré patterns shown according to an exemplary embodiment of the present disclosure. Detailed implementation manners
[0031] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0032] The terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms "a", "the" and "said" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.
[0033] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".
[0034] When the spatial frequency of the image sensor pixels is close to the spatial frequency of the stripes in the acquired image, a wavy interference pattern may be generated in the image. This wavy interference pattern is called moiré.
[0035] For example, the stripes in the image can be regular stripes on a carpet or regular stripes on the steps of an escalator.
[0036] Moreover, since white light is a composite light of multiple colors, when moiré appears in the image, false colors of colored stripes will also appear.
[0037] Figure 1A FIG. is a schematic diagram of an image containing moiré shown according to an embodiment of the present disclosure. Figure 1B FIG. is a schematic diagram of an image not containing moiré shown according to an embodiment of the present disclosure.
[0038] As Figure 1A and Figure 1B shown, by Figure 1A comparing with Figure 1B , it can be found that a curved wavy interference pattern appears on the steps of the escalator in Figure 1A , and this pattern is obviously not the stripes existing on the steps of the escalator. This pattern is the moiré generated when the spatial frequency of the image sensor pixels is close to the spatial frequency of the stripes in the acquired image. In a similar scenario, moiré may also appear in the image in the form of colored stripes, and the present disclosure will not show this in detail.
[0039] In the related art, moiré can be removed by adding a low-pass filter lens or using image filtering techniques on the acquired image. However, these methods of removing moiré will result in loss of image details.
[0040] To solve the above technical problems, the present disclosure proposes a method for eliminating moiré.
[0041] Figure 2FIG. 0 is a schematic flowchart of a method for eliminating moiré patterns according to an embodiment of the present disclosure. The method for eliminating moiré patterns can be executed by a terminal. The method for eliminating moiré patterns can be used to eliminate moiré patterns in a target image collected by an image sensor. The terminal includes, but is not limited to, communication devices such as mobile phones, tablet computers, wearable devices, sensors, and Internet of Things devices.
[0042] In some embodiments, left pixel points and right pixel points are deployed on the image sensor. The left pixel points are used to collect left-side light to generate left image data when the image sensor collects a target image, and the right pixel points are used to collect right-side light to generate right image data when the image sensor collects a target image.
[0043] As Figure 2 shown, the method for eliminating moiré patterns includes:
[0044] In step S201, obtain a left image data image block and a right image data image block corresponding to at least a part of the target image;
[0045] In step S202, determine whether moiré patterns exist in the at least a part of the region according to the defocus values of the left image data image block and the right image data image block;
[0046] In step S203, when moiré patterns exist in the at least a part of the region, eliminate the moiré patterns on the target image for the at least a part of the region.
[0047] In some embodiments, the image sensor can collect a target image, and the method for eliminating moiré patterns proposed in the present disclosure is exactly to eliminate the moiré patterns that appear on the target image.
[0048] In some embodiments, an opaque barrier is provided between the left pixel points and the right pixel points on the image sensor. The barrier is used to block the right-side light from entering the left pixel points and block the left-side light from entering the right pixel points.
[0049] In some embodiments, the left image data and the right image data corresponding to the target image can be divided into multiple image blocks.
[0050] For example, if the left image data and the right image data are respectively divided into N*M image blocks (blocks), the image blocks of the left image data and the right image data at the same coordinate position correspond to the same part of the target image.
[0051] After the left image data and the right image data are segmented, it is possible to determine whether moiré patterns exist in each image block by analyzing block by block, so as to determine all the regions in the entire target image where moiré patterns exist.
[0052] In some embodiments, an image block determined to have moiré can be further divided into finer-grained blocks.
[0053] By further dividing the image block with moiré, it is possible to further determine which part of the image block the moiré is in. Further, under the condition that the computing power permits, through more detailed division, the specific pixels with moiré can be determined.
[0054] The finer the division of the image block, the more accurate the determined position of the moiré, and the less interference will be caused to the content of other areas of the target image when eliminating the moiré subsequently.
[0055] In some embodiments, the defocus value can be determined based on the corresponding left image data block and right image data block.
[0056] Further, according to the defocus values of the left image data block and the right image data block, it is determined whether there is moiré in at least a part of the area.
[0057] In the case where there is no moiré in at least a part of the area, it is not necessary to eliminate the moiré in at least a part of the target image; in the case where there is moiré in at least a part of the area, the moiré on the target image is eliminated for at least a part of the area.
[0058] On the image sensor of the present disclosure, left pixel points and right pixel points are deployed. When acquiring a target image, the left image data and the right image data corresponding to the target image can be acquired based on the left pixel points and the right pixel points. By performing moiré detection on the left image data block and the right image data block corresponding to at least a part of the target image, it is possible to determine whether there is moiré in at least a part of the area through the defocus value, and in the case where it is determined that there is moiré, only the moiré in at least a part of the area is eliminated. The method for eliminating moiré proposed by the present disclosure can determine the position of the moiré on the target image, and when eliminating the moiré, only the position where the moiré is located is eliminated, which can avoid the loss of picture details caused by eliminating the entire target image, thereby more accurately eliminating the moiré while ensuring the picture details of the target image.
[0059] In some embodiments, the defocus values of the left image data block and the right image data block are determined by the following method: determining the variation data of the similarity between the left image data block and the right image data block with respect to the interval distance, wherein at least one interval distance corresponding to the maximum value of the similarity is the defocus value.
[0060] In some embodiments, the similarity between the left data image block and the right data image block can be calculated by the SAD (Sum of absolute differences) algorithm.
[0061] The specific formula is:
[0062] cost SAD = ∑|I L (j) - I R (j)|
[0063] where I L (j) is the left data image block corresponding to the J region of the target image, and I R (j) is the right data image block corresponding to the J region of the target image, and cost SAD is the matching cost.
[0064] In some embodiments, the right data image block can be horizontally shifted left and right relative to the left data image block, and the values of cost corresponding to the respective interval distances between the right data image block and the left data image block can be calculated SAD , so as to obtain the variation data of the similarity with respect to the interval distance.
[0065] where the value of the interval distance k between the right data image block and the left data image block ranges from (-h, +h), and the value of h is determined by the stroke range of the focusing motor of the image sensor.
[0066] Figure 3 is a schematic diagram showing the variation data of the similarity with respect to the interval distance according to an embodiment of the present disclosure.
[0067] As Figure 3 shown, the abscissa k is the interval distance between the right data image block and the left data image block, and the ordinate Cost is the value of the similarity between the right data image block and the left data image block at each interval distance. Figure 3 The peak point, that is, the maximum point, of the curve shown in
[0068] Figure 4 is a schematic diagram showing the variation data of the similarity with respect to the interval distance according to an embodiment of the present disclosure.
[0069] In some embodiments, there may be multiple maximum points in the variation data of the similarity between the right data image block and the left data image block with respect to the interval distance, and the abscissas corresponding to these maximum points are all defocus values.
[0070] As Figure 4 shown, inFigure 4 There are 4 peak points, that is, 4 maximum points, and the abscissas corresponding to these 4 maximum points are defocus values, that is, Figure 4 there are 4 defocus values in the corresponding image block.
[0071] In some embodiments, the interval distance corresponding to the maximum similarity value can also be used as the defocus value.
[0072] In some embodiments, the defocus value in the data of the change of similarity with the interval distance can be determined by the following method: when the similarity value of the right image data block at any interval distance is greater than the similarity values at the previous position and the next position of the any interval distance, determine the any interval distance as the defocus value corresponding to the maximum point.
[0073] For example, let any interval distance be i, and the corresponding similarity value be cost(i). If cost(i - 1) < cost(i) < cost(i + 1), then cost(i) is the maximum point, and the corresponding interval distance i is the defocus value.
[0074] In some embodiments, determining whether moiré exists in at least a part of the region according to the defocus values of the left image data block and the right image data block includes: when the number of defocus values is equal to 1, determining that there is no moiré in at least a part of the region; when the number of defocus values is greater than 1, determining that there is moiré in at least a part of the region.
[0075] When moiré exists in at least a part of the region, the right image data block and the left image data block may exhibit relatively high similarity at multiple interval distances. Therefore, it is possible to determine whether moiré exists in the image block by determining the number of cases where the left image data block and the right image data block have relatively high similarity.
[0076] In some embodiments, when the number of defocus values is equal to 1, it is determined that there is no moiré in at least a part of the region.
[0077] For example Figure 3 in the shown embodiment, there is a unique defocus value for the left image data block and the right image data block. Therefore, there is no moiré in the corresponding at least a part of the region.
[0078] In some embodiments, when the number of defocus values is greater than 1, it is determined that there is moiré in at least a part of the region.
[0079] For example Figure 4 in the shown embodiment, there are 4 defocus values for the left image data block and the right image data block. Therefore, there is moiré in the corresponding at least a part of the region.
[0080] In addition to determining whether moiré exists on at least a part of the area by the number of defocus values in the above embodiments, the present disclosure also provides another embodiment for determining whether moiré exists according to the defocus value.
[0081] Figure 5 It is a schematic flowchart of a method for determining whether moiré exists on at least a part of the area shown according to an embodiment of the present disclosure.
[0082] As Figure 5 shown, in some embodiments, determining whether moiré exists on at least a part of the area according to the defocus values of the left image data block and the right image data block includes:
[0083] In step S501, determine the position of the focus motor when the image sensor captures the target image;
[0084] In step S502, determine the focus position of at least a part of the area according to the defocus value and the position of the focus motor;
[0085] In step S503, if the focus position of at least a part of the area corresponding to any defocus value exceeds the stroke range of the focus motor, it is determined that moiré exists on at least a part of the area.
[0086] In some embodiments, if there is no moiré in the target image, the focus position of at least a part of the area determined according to the defocus value should be within the stroke range of the focus motor.
[0087] However, since moiré is not a characteristic of the object being photographed, the contrast of moiré is large and it is easy to interfere with the magnitude of the defocus value. Under the interference of moiré, the defocus value corresponding to the image block with moiré will be much larger than the defocus value of the same image block without moiré, which will cause the determined focus position to exceed the stroke range of the focus motor.
[0088] In some embodiments, determine the focus position of at least a part of the area according to the defocus value and the position of the focus motor.
[0089] It can be determined by the following formula:
[0090] PD_target = Current_pos + PD_value * DCC
[0091] where PD_target is the focus position of the image block corresponding to at least a part of the area, Current_pos is the position of the focus motor when capturing the target image, PD_value is the defocus value, and DCC is the unit conversion for the unit of the defocus value and the unit of the focus position.
[0092] In some embodiments, at least a portion of the area may have multiple defocus values and correspondingly have multiple in-focus positions, wherein when any in-focus position exceeds the travel range of the focus motor, it can be determined that moiré patterns exist on the at least a portion of the area.
[0093] The travel range of the focus motor can be represented by OTP_range. Whether moiré patterns exist can be determined by comparing the in-focus position PD_target corresponding to any defocus value with the travel range OTP_range of the focus motor.
[0094] For example, Figure 3 In the embodiment shown, there is only one defocus value, and the existence of moiré can be determined by simply determining that the in-focus position corresponding to the defocus value exceeds the travel range of the focus motor. Figure 4 In the illustrated embodiment, when there is more than one defocus value, the presence of moiré patterns can be determined by simply determining that the in-focus position corresponding to any defocus value exceeds the travel range of the focus motor.
[0095] In some embodiments, if there are multiple defocus values in at least a portion of the area, the maximum defocus value among the multiple defocus values is determined, and when the in-focus position corresponding to the maximum defocus value exceeds the travel range of the focus motor, it is determined that moiré exists; when the in-focus position corresponding to the maximum defocus value does not exceed the travel range of the focus motor, it is determined that no moiré exists.
[0096] If moiré patterns exist, the calculated defocus value will be biased larger due to the interference of the moiré patterns. Therefore, when there are multiple defocus values, it is possible to determine whether moiré patterns exist simply by comparing the in-focus position corresponding to the maximum defocus value and the formation range of the focus motor.
[0097] It should be noted that in the above embodiment, whether moiré patterns exist is determined by the number of defocus values, and whether moiré patterns exist is determined by comparing the in-focus position with the travel range of the focus motor. These two embodiments are independent of each other and can be implemented separately.
[0098] In some embodiments, both the method of determining whether moiré exists by the number of defocus values in the above embodiment and the method of determining whether moiré exists by comparing the in-focus position with the travel range of the focus motor in the above embodiment can be used.
[0099] In order to ensure that the positions of all moiré patterns on the target image are determined, the determination results of the above two methods may be combined to determine that at least a portion of the area determined to have moiré patterns by any one method has moiré patterns.
[0100] In some embodiments, when it is determined that moiré exists in at least a part of the region based on the defocus value, if the target image is not in a moiré scenario, the determination result may be misjudged.
[0101] Therefore, in order to determine whether the determination result that moiré exists based on the defocus value is reliable, it is also possible to confirm whether the target image is in a moiré scenario.
[0102] Figure 6 It is a schematic flowchart of a method for eliminating moiré shown according to an embodiment of the present disclosure.
[0103] As Figure 6 shown, in some embodiments, before eliminating the moiré on the target image for the at least a part of the region, the method further includes:
[0104] In step S601, obtain a preview image of the previous frame of the target image collected by the image sensor;
[0105] In step S602, when the defocus position difference of the image blocks at any same position of the target image and the preview image is greater than a preset abnormal threshold, determine that the image blocks at the any same position are abnormal;
[0106] In step S603, determine the number of abnormal image blocks of the target image and the preview image;
[0107] In step S604, confirm the determination result of whether moiré exists in the at least a part of the region; wherein, when the number of the image blocks is greater than the moiré confirmation threshold, confirm the determination that moiré exists in the at least a part of the region; when the number of the image blocks is not greater than the moiré confirmation threshold, cancel the determination that moiré exists in the at least a part of the region.
[0108] Among them, the calculation of the defocus position can refer to other embodiments of the present disclosure and will not be elaborated here.
[0109] In some embodiments, if there is moiré in the target image, there will be a large change in the defocus position at the same position between the target image and the preview image of the previous frame, and the similarity between the two images is low.
[0110] In some embodiments, if the number of abnormal image blocks is greater than the moiré confirmation threshold, it can be determined that there is moiré in the target image, and it is determined that there is moiré in at least a part of the region; if the number of abnormal image blocks is not greater than the moiré confirmation threshold, it can be determined that there is no moiré in the target image, the determination that there is moiré in at least a part of the region is cancelled, and it is determined that the determination that there is moiré in at least a part of the region is a misjudgment based on the defocus value.
[0111] Through this embodiment, it can be confirmed whether there is a misjudgment in the determination result that there is moiré in at least a part of the region, thereby avoiding the loss of picture details caused by moiré elimination of the target image without moiré in the case of misjudgment.
[0112] Figure 7 FIG. is a schematic diagram of an image processing process shown according to an embodiment of the present disclosure.
[0113] In some embodiments, the processing of the target image includes three stages, which are, in sequence: processing in the original image domain, processing in the RGB domain, and processing in the YUV domain.
[0114] As Figure 7 In the process shown by the solid line in FIG., taking the RAW domain as an example in the original image domain, the processing of the target image in the RAW domain includes, in sequence: RAW image input, preprocessing, lens shading correction, AWB correction; among them, the preprocessing may include, but is not limited to: dead pixel correction, black level correction, digital gain.
[0115] After processing in the RAW domain, the target image is processed in the RGB domain; for example, it may include, but is not limited to: demosaicing, CCM correction, Gamma correction, color space conversion.
[0116] After processing in the RGB domain, the target image is processed in the YUV domain; it includes, in sequence: YUV preprocessing, moiré elimination, subsequent processing and output of the image; among them, the YUV preprocessing may include, but is not limited to: noise reduction, tone mapping, color processing; the subsequent processing may include, but is not limited to: image encoding, output of JPEG image.
[0117] In the related art, when eliminating moiré in the YUV domain, since the target image has undergone processing in the RGB domain and YUV domain preprocessing, it may be interfered by intermediate modules. For example, the relevant modules for color processing may aggravate the false color of moiré, and the relevant modules for sharpening processing may aggravate the moiré phenomenon.
[0118] The following embodiments proposed by the present disclosure can solve the above technical problems.
[0119] As Figure 7The process shown by the dashed line. In some embodiments, determining whether moiré exists on at least a part of the region includes: determining whether moiré exists in the original image domain; and eliminating moiré on the target image for at least a part of the region includes: eliminating moiré for at least a part of the region in the YUV domain.
[0120] In some embodiments, it is possible to determine whether moiré exists in the original image domain and determine the position of the moiré in the target image; when eliminating moiré in the YUV domain, the position of the moiré determined in the original image domain in the target image is sent to the moiré elimination module, so that the module can eliminate the moiré at the position of the moiré in the target image.
[0121] Since the original image domain is at the forefront of image processing, determining the position of the moiré at this time can exclude the influence of other image processing modules on the moiré, enabling more accurate detection of the moiré. Based on the position of the existing moiré determined in the original image domain, then eliminating the moiré in the YUV domain makes the targeting stronger and the elimination of the moiré more accurate, without being interfered by other modules in image processing during moiré elimination.
[0122] In some embodiments, eliminating moiré on the target image for at least a part of the region includes at least one of the following: performing sharpening processing on the moiré alone for at least a part of the region; removing the pseudo-color information on the moiré for at least a part of the region; reducing the moiré phenomenon for at least a part of the region through filtering.
[0123] In Figure 7 In the illustrated embodiment, the moiré elimination module may further include: a sharpening module, a pseudo-color removal module, and a filtering module.
[0124] These three modules can all receive the position information of the moiré in the determined target image, and then perform targeted processing on the positions where moiré exists. Therefore, it will not affect the non-moiré regions, avoiding damage to other regions during moiré elimination processing, and thus preventing the loss of details and colors in the picture.
[0125] The following is a further introduction to these three modules.
[0126] A switch can be set in the sharpening module to determine whether to perform separate processing on the position where the moiré is located. If separate processing is performed on the position where the moiré is located, the sharpening degree of the position where the moiré is located can be reduced to achieve the purpose of reducing the moiré. It is also possible to choose not to perform separate processing on the position where the moiré is located and process the position where the moiré is located together with other regions of the target image.
[0127] The false color removal module can remove the false color information on the moiré pattern. When the false color removal module is set in the YUV domain, the false color removal module can process only the U and V color channels. The bilateral filter can be used to process the U channel and the V channel to enhance the filtering effect of the bilateral filter, thereby removing the false color, or the method of reducing saturation can be adopted to reduce the saturation of the false color to achieve the purpose of removing the false color.
[0128] The filtering module can alleviate the moiré pattern phenomenon. The low-pass filter, bilateral filter, etc. can be used to process the position where the moiré pattern is located. When the filtering module is set in the YUV domain, the filtering module can process only the Y channel, that is, the luminance channel. For example, the intensity of the low-pass filtering can be enhanced and the sharpness of the moiré pattern can be weakened to achieve the purpose of alleviating or eliminating the moiré pattern.
[0129] The present disclosure can determine the area where the moiré pattern exists through the defocus value, and specifically eliminate the moiré pattern in the area where the moiré pattern exists, avoiding the loss of details caused by eliminating the moiré pattern in other areas of the target image, and being able to more accurately remove the moiré pattern on the target image.
[0130] Corresponding to the embodiment of the method for eliminating moiré pattern of the present disclosure, the present disclosure also provides an embodiment of the corresponding device for eliminating moiré pattern.
[0131] Left pixel points and right pixel points are deployed on the image sensor. The left pixel points are used to collect left-side light to generate left image data when the image sensor collects the target image, and the right pixel points are used to collect right-side light to generate right image data when the image sensor collects the target image.
[0132] Please refer to Figure 8 , Figure 8 which is a block diagram of the device for eliminating moiré pattern in an embodiment of the present disclosure. As Figure 8 shown, the device for eliminating moiré pattern includes:
[0133] An acquisition unit 810, configured to acquire a left image data block and a right image data block corresponding to at least a part of the area of the target image;
[0134] A determination unit 820, configured to determine whether there is a moiré pattern on the at least a part of the area according to the defocus values of the left image data block and the right image data block;
[0135] An elimination unit 830, configured to eliminate the moiré pattern on the target image for the at least a part of the area when there is a moiré pattern on the at least a part of the area.
[0136] In some embodiments, determining whether moiré exists in at least a part of the region according to the defocus values of the left image data block and the right image data block includes: when the number of the defocus values is equal to 1, determining that no moiré exists in at least a part of the region; when the number of the defocus values is greater than 1, determining that moiré exists in at least a part of the region.
[0137] In some embodiments, determining whether moiré exists in at least a part of the region according to the defocus values of the left image data block and the right image data block includes: determining the position of the focusing motor when the image sensor captures the target image; determining the focus position of at least a part of the region according to the defocus value and the position of the focusing motor; if the focus position of at least a part of the region corresponding to any defocus value exceeds the travel range of the focusing motor, determining that moiré exists in at least a part of the region.
[0138] In some embodiments, the defocus values of the left image data block and the right image data block are determined by the following method: determining the change data of the similarity between the left image data block and the right image data block with the interval distance, where at least one interval distance corresponding to the maximum value of the similarity is the defocus value.
[0139] In some embodiments, before eliminating moiré on the target image for at least a part of the region, the device is further configured to: obtain a preview image of the previous frame of the target image captured by the image sensor; when the difference in the focus positions of the image blocks at any same position between the target image and the preview image is greater than a preset abnormal threshold, determining that the image block at any same position is abnormal; determining the number of image blocks with abnormalities in the target image and the preview image; confirming the determination result of whether moiré exists in at least a part of the region; where when the number of the image blocks is greater than the moiré confirmation threshold, confirming the determination that moiré exists in at least a part of the region; when the number of the image blocks is not greater than the moiré confirmation threshold, canceling the determination that moiré exists in at least a part of the region.
[0140] In some embodiments, the processing of the target image includes three stages, which are, in sequence: processing in the original image domain, processing in the RGB domain, and processing in the YUV domain; where determining whether moiré exists in at least a part of the region includes: determining whether moiré exists in the original image domain; eliminating moiré on the target image for at least a part of the region includes: eliminating moiré for at least a part of the region in the YUV domain.
[0141] In some embodiments, eliminating moiré patterns on the target image for at least a part of the regions includes at least one of the following: performing sharpening processing on the moiré patterns for at least a part of the regions alone; removing pseudo-color information on the moiré patterns for at least a part of the regions; reducing the moiré pattern phenomenon for at least a part of the regions by filtering.
[0142] For the specific implementation process of the functions and roles of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method, which will not be elaborated here.
[0143] An embodiment of the present disclosure also provides an electronic device, including: a processor and a memory; the memory is used for storing a computer program; the processor is used for executing the method for eliminating moiré patterns according to any one of the above embodiments by calling the computer program.
[0144] An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and characterized in that the program, when executed by a processor, implements the method for eliminating moiré patterns according to any one of the above embodiments.
[0145] Figure 9 FIG. 13 is a schematic block diagram of a device 900 for eliminating moiré patterns according to an embodiment of the present disclosure. For example, the device 900 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0146] Refer to Figure 9 , the device 900 may include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.
[0147] The processing component 902 generally controls the overall operation of the device 900, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above information receiving method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.
[0148] The memory 904 is configured to store various types of data to support the operation of the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, and the like. The memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.
[0149] The power supply component 906 provides power to various components of the device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.
[0150] The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
[0151] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC) that is configured to receive external audio signals when the device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 further includes a speaker for outputting audio signals.
[0152] The I / O interface 912 provides an interface between the processing component 902 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power-on button, and a lock button.
[0153] The sensor assembly 914 includes one or more sensors for providing a status assessment of various aspects of the device 900. For example, the sensor assembly 914 can detect the on / off state of the device 900, the relative positioning of components, such as the display and keypad of the device 900, the sensor assembly 914 can also detect a change in the position of the device 900 or a component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900, and the temperature change of the device 900. The sensor assembly 914 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0154] The communication component 916 is configured to facilitate communication between the device 900 and other devices in a wired or wireless manner. The device 900 can access a wireless network based on communication standards, such as WiFi, 2G, 3G, 4G LTE, 5G NR, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0155] In an exemplary embodiment, the device 900 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above information receiving method.
[0156] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the above instructions can be executed by a processor 920 of the device 900 to complete the above information receiving method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.
[0157] Other embodiments of the present disclosure will be readily apparent to those skilled in the art in view of the specification and practice of the disclosure herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only illustrative, and the true scope and spirit of the present disclosure are pointed out by the following claims.
[0158] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.
[0159] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.
[0160] The methods and apparatuses provided by the embodiments of the present disclosure have been introduced in detail above. Specific examples are used herein to illustrate the principles and implementation manners of the present disclosure. The description of the above embodiments is only used to help understand the method and its core idea of the present disclosure; at the same time, for those of ordinary skill in the art, according to the idea of the present disclosure, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present disclosure.
Claims
1. A method for eliminating moiré patterns, characterized in that, Left pixel points and right pixel points are deployed on an image sensor. The left pixel points are used to collect left-side light when the image sensor collects a target image to generate left image data, and the right pixel points are used to collect right-side light when the image sensor collects the target image to generate right image data. The method includes: Obtain a left image data block and a right image data block corresponding to at least a part of the target image; Determine whether moiré patterns exist in the at least a part of the region according to the defocus values of the left image data block and the right image data block; When moiré patterns exist in the at least a part of the region, eliminate the moiré patterns on the target image for the at least a part of the region.
2. The method according to claim 1, characterized in that, The determining whether moiré patterns exist in the at least a part of the region according to the defocus values of the left image data block and the right image data block includes: When the number of the defocus values is equal to 1, determine that no moiré patterns exist in the at least a part of the region; When the number of the defocus values is greater than 1, determine that moiré patterns exist in the at least a part of the region.
3. The method according to claim 1, wherein The determining whether moiré patterns exist in the at least a part of the region according to the defocus values of the left image data block and the right image data block includes: Determine the position of the focus motor when the image sensor collects the target image; Determine the quasi-focus position of the at least a part of the region according to the defocus value and the position of the focus motor; If the quasi-focus position of the at least a part of the region corresponding to any defocus value exceeds the stroke range of the focus motor, determine that moiré patterns exist in the at least a part of the region.
4. The method according to claim 1, wherein The defocus values of the left image data block and the right image data block are determined by the following method: Determine the variation data of the similarity between the left image data block and the right image data block with the interval distance, where at least one interval distance corresponding to the maximum value of the similarity is the defocus value.
5. The method according to claim 2 or 3, characterized in that, Before eliminating the moiré patterns on the target image for the at least a part of the region, the method further includes: Obtain a preview image of the previous frame of the target image collected by the image sensor; When the difference in the quasi-focus positions of the image blocks at any same position of the target image and the preview image is greater than a preset abnormal threshold, determine that the image blocks at the any same position are abnormal; Determine the number of the abnormal image blocks of the target image and the preview image; Confirm the determination result of whether moiré patterns exist in the at least a part of the region; where When the number of the image blocks is greater than the moiré confirmation threshold, confirm the determination that moiré patterns exist in the at least a part of the region; When the number of the image blocks is not greater than the moiré confirmation threshold, cancel the determination that moiré patterns exist in the at least a part of the region.
6. The method according to claim 1, wherein The processing of the target image includes three stages, which are: processing in the original image domain, processing in the RGB domain, and processing in the YUV domain; where The determining whether moiré patterns exist in the at least a part of the region includes: determining whether moiré patterns exist in the original image domain; Performing moiré pattern elimination on the target image for the at least a part of the region includes: performing moiré pattern elimination on the at least a part of the region in the YUV domain.
7. The method according to claim 1, wherein Performing moiré pattern elimination on the target image for the at least a part of the region includes at least one of the following: Performing sharpening processing on the moiré pattern alone for the at least a part of the region; Removing the false color information on the moiré pattern for the at least a part of the region; Reducing the moiré pattern phenomenon for the at least a part of the region by filtering.
8. A moiré pattern elimination device, characterized in that, Left pixel points and right pixel points are deployed on an image sensor. The left pixel points are used to collect left-side light to generate left image data when the image sensor collects a target image, and the right pixel points are used to collect right-side light to generate right image data when the image sensor collects a target image. The apparatus includes: An acquisition unit configured to acquire a left image data image block and a right image data image block corresponding to at least a part of the region of the target image; A determination unit configured to determine whether there is a moiré pattern on the at least a part of the region according to the defocus values of the left image data image block and the right image data image block; An elimination unit configured to, when there is a moiré pattern on the at least a part of the region, perform moiré pattern elimination on the target image for the at least a part of the region.
9. An electronic device, characterized in that, including: A processor and a memory; The memory is used to store a computer program; The processor is used to execute the moiré pattern elimination method according to any one of claims 1-7 by calling the computer program.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the moiré pattern elimination method according to any one of claims 1-7.